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Abstract. We study the Torelli locus Tg in the moduli space Ag of abelian 
varieties. We consider special subvarieties (Shimura subvarieties) contained 
in the Torelli locus. We review the construction of some non-trivial examples, 
and wo discuss some conjectures, techniques and recent progress. 



Introduction 

The Torelli morphism 

An algebraic curve C has a Jacobian Jc = Pic°(C); which comes equipped 
with a canonical principal polarization A: Jc — > Jc- The dimension of Jc equals 
the genus g of the curve. One should like to understand Jc knowing certain 
properties of C, and, conversely, we like to obtain information about C from 
properties of Jc or the pair ( Jc, A). 

As an important tool for this we have the moduli spaces of algebraic curves 
and of polarized abelian varieties. The construction that associates to C the 
principally polarized abelian variety {Jc, A) defines a morphism 

J- Mg ^ Ag, 

the Torelli morphism, which by a famous theorem of Torelli is injective on geo- 
metric points. (Here we work with the coarse moduli schemes.) The image 
T° := j(Mg) is called the (open) Torelli locus, also called the Jacobi locus. Its 
Zariski closure Tg inside Ag is called the Torelli locus. We should like to study the 
interplay between curves and their Jacobians using the geometry of Mg and of Ag, 
and relating these via the inclusion Tg C Ag. 

Special subvarieties 

Although the moduli spaces Mg and Ag have been studied extensively, it 
turns out that there are many basic questions that are still open. The moduli 
space Ag locally has a group-like structure, in a sense that can be made precise. 
As a complex manifold, Ag(C) is a quotient of the Siegel space, which is a princi- 
pally homogeneous space under the group CSp2g(IR). Hence it is natural to study 



2000 Mathematics Subject Classification, Primary 14G35, 14H, 14K; Secondary: IIG. 
Key words and phrases. Curves and their Jacobians, Torelli locus, Shimura varieties. 



2 



The Torelli locus and special subvarieties 



subvaricties that appear as images of orbits of an algebraic subgroup; such a sub- 
variety is called a special subvariety. (We refer to Section [3] for a more precise 
definition of this notion.) 

The zero dimensional special subvarieties are precisely the CM points, cor- 
responding to abelian varieties where End(A) Q contains a commutative semi- 
simple algebra of rank 2-dim(A). These are fairly well understood. The arith- 
metical properties of CM points form a rich and beautiful subject, that plays an 
important role in the study of the moduli space. The presence of a dense set of 
CM points enabled Shimura to prove for a large class of Shimura varieties that 
they admit a model over a number field. In its modern form this theory owes much 
to Deligne's presentation in [21] and [l^, and the existence of canonical models 
for all Shimura varieties was established by Borovoi and Milne. 

For the problems we want to discuss, the special points again play a key role. 
It is not hard to show that on any special subvariety of Ag over C the special 
points are dense, even for the analytic topology. A conjecture by Y. Andre and 
F. Oort says that, conversely, an algebraic subvariety with a Zariski dense set of 
CM points is in fact a special variety. See 13.181 There is a nice analogy between 
this conjecture and the Manin-Mumford conjecture, proven by Raynaud, which 
says that an algebraic subvariety of an abelian variety that contains a Zariski 
dense set of torsion points is the translate of an abelian subvariety under a torsion 
point. 

Klingler and Yafaev, using work of Ullmo and Yafaev, have announced a 
proof of the Andre-Oort conjecture under assumption of the Generalized Riemann 
Hypothesis for CM fields. 

Coleman's Conjecture and special subvarieties in the Torelli locus 

In view of the above, it is natural to investigate the subvariety Tg C Ag from 
the perspective of special subvarieties of Ag. A conjecture by Coleman, again based 
on the analogy with the Manin-Mumford conjecture, says that for 5 > 3 there are 
only finitely many complex curves of genus g such that Jc is an abelian variety 
with CM. See 14.11 In terms of moduli spaces, this says that for 5 > 3 the number 
of special points on T° should be finite. Combining this with the Andre-Oort 
conjecture, it suggests that for large g there are no special subvarieties Z C Ag of 
positive dimension that are contained inside Tg and meet T"; see 14.21 We should 
like to point out that there is no clear evidence for this expectation, although there 
are several partial results in support of it. See for instance Theorem 14.71 below, 
which gives a weaker version of Coleman's conjecture. 

As we shall explain in Section [51 Coleman's Conjecture is not true for g G 
{4, 5, 6, 7}. The reason is that for these genera we can find explicit families of curves 
such that the corresponding Jacobians trace out a special subvariety of positive 
dimension in Ag. All known examples of this kind, for <? > 4, arise from families 
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of abelian covers of P^, where we fix the Galois group and the local nionodromies, 
and we vary the branch points. For cyclic covers it can be shown that, beyond 
the known examples, this construction gives no further families that give rise to a 
special subvariety in A^; see Theorem 15. 171 

In this paper, we give a survey of what is known about special subvarieties 
in the Torelli locus, and we try to give the reader a feeling for the difficulties 
one encounters in studying these. Going further, one could ask for a complete 
classification of such special subvarieties Z C Tg for every g. We should like to 
point out that already for g = 4 we do not have such a classification, in any sense. 
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0.1. Notation 

If G is a group scheme of finite type over a field fc, we denote its identity 
component by G". In case fc is a subfield of M we write G(]R)"'" for the identity 
component of G(R) with regard to the analytic topology. If G is reductive, G^'^ 
denotes the adjoint group. 

The group of symplectic similitudes. Let a positive integer g be given. Write 
V := Z^^, and let (j>: V x V Z he the standard symplectic pairing, so that on 
the standard ordered basis {ei}i=i^,,,^2g we have (f>(ei,ej) = 0ifi + j^2g+l 
and 0(ei, e2g+i_i) = 1 if z < The group CSp(y, 0) is the reductive group 
over Z of symplectic similitudes of V with regard to the form 0. If there is no 
risk of confusion we simply write CSp2g for this group. We denote by ly : CSp2g ^• 
Gm the multiplier character. The kernel of v is the symplectic group Sp2g of 
transformations of V that preserve the form (j). 
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1. The Torelli morphism 
1.1. Moduli of abelian varieties 

Fix an integer <? > 0. If 5* is a base scheme and A is an abelian scheme over S, 
we write ^* for the dual abelian scheme. There is a canonical homomorphism 
ka - A ^ A** , which Cartier and Nishi proved to be an isomorphism; see [5] , [TO] ; 
[Si]; also see [50], Theorem 20.2. 

On ^ Xg^* we have a Poincare bundle V. A polarization of ^ is a symmetric 
isogcny A : A — > A* with the property that the pull-back of V via the morphism 
(id, A) : A ^ A Xs is a. relatively ample bundle on A over S. The polariza- 
tion A is said to be principal if in addition it is an isomorphism of abelian schemes 
over S. A pair (A, A) consisting of an abelian scheme over S together with a prin- 
cipal polarization is called a principally polarized abelian scheme. Wc abbreviate 
"principally polarized abelian scheme(s) /variety/varieties" to "ppav" (sic!). 

We denote by Ag the moduli stack over Spec(Z) of (^-dimensional ppav. It is 
a connected Deligne-Mumford stack that is quasi- projective and smooth over Z, 
of relative dimension g{g + l)/2. The characteristic zero fiber Ag^q can be shown 
to be geometrically irreducible by transcendental methods; cf Section Using 
this, Chai and Faltings have proved that the characteristic p fibers are geomet- 
rically irreducible, too; see [IH]: [12: and [3D] Chap. IV, Corollary 6.8. A pure 
characteristic p proof can be obtained using the results of |65] . 

Note that later in this article we shall use the same notation Ag for the 
moduli stack over some given base field. 

For many purposes it is relevant to consider ppav together with a level struc- 
ture. Wc introduce some notation and review some facts. Let again (A, A) be a 
ppav of relative dimension g over some base scheme S. Given m £ Z we have a 
"multiplication by m" endomorphism [m] : A A. We denote by A[m] the kernel 
of this endomorphism, in the scheme-theoretic sense. Assume that m 0. Then 
A[m] is a finite locally free group scheme of rank m^^. There is a naturally defined 
bilinear pairing 

e,^„: A[m] x A[m] ^i„i , 

called the Weil pairing, that is non-degenerate and that is symplectic in the sense 
that e^(x, x) = 1 for all ^-schemes T and all T-valucd points x £ A[m](T). If m 
is invertible in T{S, Os) then the group scheme A[m] is etale over S. 

Fix an integer m > 1. Write Z[Cm] for Z[t]/($m), where G Z[t] is the 
mth cyclotomic polynomial. We shall first define a Deligne-Mumford stack A'^ 
over the spectrum of the ring Rm '■= Z[Cm, 1/m]. The moduli stack ^g,[,„] is then 
defined to be the same stack, but now viewed as a stack over Z[l/m], with as 
structural morphism the composition 



■^'g,[rn] ^ SpCc(i?„0 ^ SpCc(Z[l/m]) 
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The reason that we want to view Ag^[„i] as an algebraic stack over Z[l/m] is that 
this is the stack whose characteristic zero fiber has a natural interpretation as a 
Shimura variety; see the discussion in 13.31 below. 

Let 5 be a scheme over R^- Note that this just means that 5' is a scheme 
such that m is invertible in T{S, Os), and that we are given a primitive mth root 
of unity C G T{S,Os)- We may view this C an isomorphism of group schemes 
(: (Z/mZ) ^ ij,,n- Let V and (j) be as defined in 10.11 Given a ppav {A,X) 
of relative dimension g over S, by a (symplectic) level m structure on (A, A) we 
then mean an isomorphism of group schemes a: (V/mV) ^ A[m\ such that the 
diagram 

{V/mV) X (V/mV) — ^ (Z/mZ) 
A[m] X A[m] — — — 

is commutative. 

We write A'^ for the moduli stack over Rm of ppav with a level m structure. 
Again this is a quasi-projective smooth Deligne-Mumford stack with geometrically 
irreducible fibers. If to > 3, it is a scheme. The resulting stack 

Ag^[rn] ■■= (-4g,[,„] ^ Spec(i?,„) -> Spec(Z[l/m])) 

is a quasi-projective smooth Deligne-Mumford stack over Z[l/m] whose geometric 
fibers have (p{m) irreducible components. 

We now assume that to > 3. The finite group Q := CSp2g(Z/mZ) acts on 
the stack A' r^i via its action on the level structures. Note that this is not an 
action over i?,„; only the subgroup Sp2g(Z/TOZ) acts by automorphisms over 
We do, however, obtain an induced action of G on ^g.[m] over Z[l/m]. The stack 
quotient of Ag^[rii] modulo Q is just Ag (g) Z[l/m]. The scheme quotient of ^gjm] 
modulo Q is the coarse moduli scheme Ag (g) Z[l/m]. For mi, m2 > 3 these coarse 
moduli schemes agree over Z[l/(mim2)]; hence we can glue them to obtain a 
coarse moduli scheme Ag over Z. 

The stacks ^gjm] sre not complete. There are various ways to compactify 
them. The basic reference for this is the book [3D] . There are toroidal compactifi- 
cations that depend on the choice of some cone decomposition S (see op. cit. for 
the details); these compactifications map down to the Baily-Borel (or minimal, or 
Satake) compactification. 

1.2. Moduli of curves 

Fix an integer g > 2. We denote by A4g the moduli stack of curves of genus g 
over Spec(Z). It is a quasi-projective smooth Deligne-Mumford stack over Z of 
relative dimension 3(7 — 3 for g > 2, with geometrically irreducible fibers. 
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Wc write Mg for the Dclignc-Mumford compactification of Mg; it is the 
stack of stable curves of genus g. See [25]. The boundary A^g \ Aig is a divisor 
with normal crossings; it has irreducible components for < i < [g/2j. There 
is a non-empty open part of Ag that parametrizes irreducible curves with a single 
node. Similarly, for i > there is a non-empty open part of A^ parametrizing 
curves with precisely two irreducible components, of genera i and g — i. The 
complement of Aq is the open substack Mg^ C Mg of curves of compact type. 
Here we recall that a stable curve C over a field k is said to be of compact type if the 
connected components of its Picard scheme are proper over k. If fc is algebraically 
closed this is equivalent to the condition that the dual graph of the curve has 
trivial homology; so the irreducible components of C are regular curves, the graph 
of components is a tree, and the sum of the genera of the irreducible components 
equals g. Note that Mg C Mf. 

For the corresponding coarse moduli schemes we use the notation Mg and M^'. 

1.3. The Torelli morphism and the Torelli locus 

Let C be a stable curve of compact type over a base scheme S, with fibers 
of genus g > 2. The Picard scheme Picc/s is a smooth separated 5- group scheme 
whose components are of finite type and proper over S. In particular, the identity 
component Jc '■= Picc/s is an abelian scheme over S. It has relative dimension g 
and comes equipped with a canonical principal polarization A. 

The functor that sends C to the pair ( Jc, A) defines a representable morphism 
of algebraic stacks 

J-.Mf-^Ag, 

called the Torelli morphism. It is known that this morphism is proper. 

Torelli's celebrated theorem says that for an algebraically closed field k the 
restricted morphism j : Aig — s- Ag is injective on fc-valued points. In other words, 
if Ci and C2 are smooth curves over k such that (Jc^, Ai) and (JC2JA2) are 
isomorphic then Ci and C'2 are isomorphic. This injectivity property of the Torelli 
morphism does not hold over the boundary; for instance, if C is a stable curve of 
compact type with two irreducible components then the Jacobian of C does not 
"see" at which points the two components are glued. For a detailed study of what 
happens over the boundary, see [8]. 

Let us now work over a field k. On coarse moduli schemes the Torelli mor- 
phism gives rise to a morphism j : M^* — > Ag. We define the Torelli locus Tg C Ag 
to be the image of this morphism. It is a reduced closed subscheme of Ag . By the 
open Torelli locus T° C Tg we mean the image of Mg, i.e., the subscheme of Tg 
whose geometric points correspond to Jacobians of nonsingular curves. Note that 
T° C Tg is open and dense. 

The boundary Tg \ T° is denoted by T^*"^. The notation refers to the fact 
that a point s € Tg(fc) is in Tg'^'^ if and only if the corresponding ppav (As, As) 
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is decomposable, meaning that we have ppav {Bi,fii) and (-62,-^2) of smaUer 
dimension such that {As,Xs) ^ {Bi,^i) x (-62,^2) ppav. In one direction this 
is clear, for if C is a reducible curve of compact type, ( Jc, A) is the product, as a 
ppav, of the Jacobians of the irreducible components of C. Conversely, if (A^, A^) 
is decomposable, it has a symmetric theta divisor that is reducible, which implies 
it cannot be the Jacobian of an irreducible (smooth and proper) curve. 

Remark 1.4. Let g > 2. The Torelli morphism j : Mg Ag is ramified at 
the hyperelliptic locus. Outside the hyperelliptic locus it is an immersion. The 
picture is different for the Torelli morphism j : Mg — > Ag on coarse moduli schemes. 
The morphism j : M^^q ^g,Q on the characteristic zero fibers is an immersion; 
however, in positive characteristic this is not true in general. See [57], Corollaries 
2.8, 3.2 and 5.3. 

2. Some abstract Hodge theory 

In this section we review some basic notions from abstract Hodge theory. We 
shall focus on examples related to abelian varieties. 

2.1. Basic definitions 

We start by recalling some basic definitions. For a more comprehensive treat- 
ment we refer to [1D| and [55] . 

Let S := Resc/R(Gin), which is an algebraic torus over M of rank 2, called 
the Deligne torus. Its character group is isomorphic to Z^, on which complex 
conjugation, the non-trivial element of Gal(C/M), acts by (7711,7712) ('772, "^i)- 
We have §(R) = R* and §(C) = C* x C*. Let it;: G^x § be the cocharacter 
given on real points by the natural embedding M* ^ C*; it is called the weight 
cocharacter. Let Nm: § — !> G,n,K be the homomorphism given on real points by 
z I— ?> zz; it is called the norm character. 

A "L-Hodge structure of weight fc is a torsion-free Z-module H of finite rank, 
together with a homomorphism of real algebraic groups h: E> —?■ GL{H)k such that 
how. Gni GL{H)r is the homomorphism given by z i-> z^*^ • idn- The Hodge 
decomposition 

He = (Sp+q=k -ffc ' 

is obtained by taking H^''^ to be the subspace of He on which (zi, Z2) G C* x C* = 
§(C) acts as multiplication by z^^'z^^. (We follow the sign convention of [24]; see 
loc. cit.. Remark 3.3.) 

As an example, the Tate structure Z(77) is the Hodge structure of weight 
— 27i with underlying Z-module Z(77) — (27ri)" • Z given by the homomorphism 
Nm" : § Gm.R = GL(Z(7i))jj. If H is any Hodge structure, we write H{n) for 

i?®zZ(77). 
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The endomorphism C — h{i) of H-r is known as the Weil operator. If i7 is a 
Z-Hodge structure of weight k, a polarization of iJ is a homomorphism of Hodge 
structures ip: H ® H — !> 1{—k) such that the bilinear form 

given by (x, y) i-> (27ri)'^ ■ ip^{x ® C{y)) is symmetric and positive definite. (Here 
'i/'R is obtained from ij) by extension of scalars to M.) This condition implies that 
ip is symmetric if k is even and alternating if k is odd. 

Instead of working with integral coefhcients, we may also consider Q-Hodge 
structures. The above definitions carry over verbatim. 

Example 2.2. Let (A, A) be a polarized complex abelian variety. The first ho- 
mology group H = Hi{A, Z) carries a canonical Hodge structure of type (—1, 0) + 
(0, —1). (By this we mean that H'^'^' — (0) for all pairs (p, q) different from (—1, 0) 
and (0,-1).) The polarization A: A — !> A* (in the sense of the theory of abelian 
varieties) induces a polarization (in the sense of Hodge theory) ij): H ®H ^ Z(l). 
Let CSp(i?, iji) C GL(7J) be the algebraic group of symplectic similitudes of H 
with respect to the symplectic form ij:. The homomorphism h:^ ^ GL(i7)R that 
gives the Hodge structure factors through CSp(i?, 'i/')R. 

A crucial fact for much of the theory we want to discuss is that the map 
{A, A) M> [H, -0) gives an equivalence of categories 

'polarized complexl eg ^ f polarized Hodge structures 1 
abelian varieties J \ of type (-1, 0) + (0, -1) J 

In 13.31 below we shall explain how this fact gives rise to a modular interpretation 
of certain Shimura varieties. As a variant, the category of abelian varieties is 
equivalent to the category of polarizable Z-Hodge structures. Another variant is 
obtained by working with Q-coefficients; we get that the category of abelian vari- 
eties up to isogeny is equivalent to the category of polarizable Q-Hodge structures 
of type (-1,0) + (0,-1). 

2.4. Hodge classes and Mumford-Tate groups 

Let _ff be a Q-Hodge structure of weight k. By a Hodge class in H we mean 
an element of 

so, a rational class that is purely of type (0, 0) in the Hodge decomposition. Clearly, 
a non-zero Hodge class can exist only if fc = 0. If the weight is and Fil'iJc is 
the Hodge filtration, the space of Hodge classes is also equal to 

H n Fil"i7c , 

where again the intersection is taken inside He- 

As an example, we note that a Q-linear map H ^ H, viewed as an element 
of EndQ(iJ) = 7?^ (SD i?, is a homomorphism of Hodge structures if and only it is 



(2.3) 
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a Hodge class for the induced Hodge structure on EndQ(iJ). (Note that ® H 
indeed has weight 0.) Similarly, a polarization H (g) H iQ(— fc) gives rise to a 
Hodge class in {H'')'^^{-k). 

Let /i: § — >■ GL{H)]^ be the homomorphism that defines the Hodge struc- 
ture on H. Consider the homomorphism {h, Nm) : S — ?> GL{H)tr x Gm.R- The 
Mumford-Tate group of H , notation MT(i7), is the smallest algebraic subgroup 
M C GL{H) X Gm over Q such that {h, Nm) factors through Mr. As we shall see 
in Example 12.51 below, in some important cases the definition takes a somewhat 
simpler form. 

It is known that MT{H) is a connected algebraic group. If the Hodge struc- 
ture H is polarizable, MT{H) is a reductive group; this plays an important role 
in many applications. 

The crucial property of the Mumford-Tate group is that it allows us to cal- 
culate spaces of Hodge classes, as we shall now explain. For simplicity we shall 
assume the Hodge structure H to be polarizable. 

We consider the Hodge structures we can build from H by taking direct 
sums, duals, tensor products and Tate twists. Concretely, given a triple m = 
{mi, 1112, m^) with mi, m2 G Z>o and ms G Z, define 

T{m) := H®""^ ® (i/V)®"^ ® ^(mg) . 

The group GL(iJ) x Gm naturally acts on H via the projection onto its first factor; 
hence it also acts on . We let it act on Q(l) through the second projection. 
This gives us an induced action of GL(i/) x Gm on T{m). The crucial property 
of the Mumford-Tate group MT(i7) C GL(i?) x Gm is that for any such tensor 
construction T = T{m), the subspace of Hodge classes T f] r°'° is precisely the 
subspace of MT(i7)-invariants. Note that the MT(ff)-invariants are just the in- 
variants in T under the action of the group MT(i?)(Q) of Q-rational points of the 
Mumford-Tate group. As the Q-points of MT{H) are Zariski dense in the C-group 
MT(i?)c, the subspace of MT(i7)(C)-invariants in T (g) C equals 

(T n T^-°) ®Q c . 

The above property of the Mumford-Tate group characterizes it uniquely. In 
other words, if M C GL(_ff ) x Gm is an algebraic subgroup with the property that 
for any T = T[m) the space of Hodge classes T fl T^'° equals the subspace of 
M-invariants in T, we have M = MT{H). (See [24], Section 3.) In this way we 
have a direct coupling between the Mumford-Tate group and the various spaces 
of Hodge classes in tensor constructions obtained from H . 

Example 2.5. Consider a polarized abelian variety {A, A) over C. Let Hq = 
Hi{A, Q) with polarization form -0 : iJQ ® Hq Q(l) be the associated Q-Hodge 
structure, and recall that the Hodge structure is given by a homomorphism h: S ^ 
CSp(i?Q, '0)r- Define the Mumford-Tate group of A, notation MT{A), to be the 
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smallest algebraic subgroup AI C CSp(i?Q,V') over Q with the property that h 
factors through Mr. 

The Mumford-Tate group as defined here is really the same as the Mumford- 
Tate group of the Hodge structure Hq. To be precise, MT(iJQ) C G'L{Hq) x Gm 
as defined above is the graph of the multiplier character v: CSp{Hq,'>p) — > Gm 
restricted to MT{A). When considering Hodge classes in tensor constructions T 
as above, we let MT{A) act on the Tate structure Q{1) through the multiplier 
character. 

If wc want to understand how the Mumford-Tate group is used, the simplest 
non-trivial examples are obtained by looking at endomorphisms. As customary 
we write End°(A) for End(A) (g) Q. By the equivalence of categories of (|2.3I) . in 
the version with (Q-cocfficients, we have 

(2.6) End°(A) ^ EndQ_Hs(i^Q) = EndQ(iJQ)"T(^) . 

Once we know the Mumford-Tate group, this allows us to calculate the endomor- 
phism algebra of A. In practice we often use this the other way around, in that 
we assume End°(A) known and we use (|2.6p to obtain information about MT{A). 

In general Hodge classes on A are not so easy to interpret geometrically; 
see for instance the example in j53j . § 4. (In this example, A is a simple abelian 
fourfold and there are Hodge classes in i?^(A^,Q) that are not in the algebra 
generated by divisor classes; see |49| and [50|, (2.5).) According to the Hodge 
conjecture, all Hodge classes should arise from algebraic cycles on A but even for 
abelian varieties this is far from being proven. 

2.7. Hodge loci 

We study the behavior of Mumford-Tate groups in families. The setup here is 
that we consider a polarizable Q-VHS (variation of Hodge structure) of weight k 
over a complex manifold S. Let H denote the underlying Q-local system. For 
each s £ <S'(C) we have a Mumford-Tate group MT^ C GL('H(s)) x Gm, and we 
should like to understand how this group varies with s. This is best described by 
passing to a universal cover tt: S ^ S. Write H := 7r*H, and let H := T^S^Ti). 
We have a trivialization —?' H x S. Using this we may view the Mumford-Tate 
group MTx of a point x G S (by which we really mean the Mumford-Tate group 
of the image of x in 5*) as an algebraic subgroup of GL{H) x Gm- So, passing to 
the universal cover has the advantage that we can describe the VHS as a varying 
family of Hodge structures on some given space H and that we may describe the 
Mumford-Tate groups MT^ as subgroups of one and the same group. 

Given a triple m = (mi, m2, ma) as in l2.4[ consider the space 

r(m) := i?®"! ® (_ffV^®m2 g5 Q(j^g) ^ 
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on which we have a natural action of GL(iJ) x Gm- We may view an element 
t e T{m) as a global section of the local system 

f{m) := ® (7iv)»m2 ^ q(to3)^ ^ 

which underlies a Q-VHS of weight (mi — ■m2)k — 2m3 on S. (The Tate twist 
Q(m3)^ is a constant local system; it only has an effect on how we index the 
Hodge filtration on T'(m)c.) In particular, it makes sense to consider the set 
Y{t) C S of points X G S for which the value tx of f at x is a Hodge class. As 
remarked earlier, we can only have nonzero Hodge classes if the weight is zero, so 
we restrict our attention to triples m = (mi, 7712,111,3) with {mi — m2)k = 2m3. 
With this assumption on m, define 

Y{t) -{xeSlt^e Fil°r(m)c(a;)} . 

As Fil°r(m)c C r(m)c is a holomorphic subbundle, Y(t) C S" is a countable 
union of closed irreducible analytic subsets of S. 

The first thing we deduce from this is that there is a countable union of proper 
analytic subsets Y, C S such that the Mumford-Tatc group MT^ is constant (in a 
suitable sense) on 5 \ S, and gets smaller if we specialize to a point in S. Let us 
make this precise. Define S C S" by 

S UY{t) 

where, with notation as introduced above, the union is taken over all triples m 
with (mi — m2)k = 2m3, and all t £ T{S, T{m)) such that Y{t) is not the whole S. 

The main point of this definition of S is the following. If .t G S\T. and tx is a 
Hodge class in the fiber of T{m) at the point x, we can extend tx in a unique way 
to a global section t of T{m) over S, and by definition of S this global section is a 
Hodge class in every fiber. By contrast, if a; G I] then there is a tensor construction 
T{m) and a Hodge class tx whose horizontal extension t is not a Hodge class in 
every fiber. 

It follows that for X € S\Y. the Mumford-Tate group MT^ C GL{H) x G,„ 
is independent of the choice of x. We call the subgroup of GL{H) x Gm thus 
obtained the generic Mumford-Tate group. Let us denote it by MT^''". Further, 
for any x € S we have an inclusion MTa, C MT^^^". 

The subset E C .5 is stable under the action of the covering group oi S/S 
and therefore defines a subset T, C S. It follows from the construction that E is a 
countable union of closed analytic subspaces of S. We say that a point s E S{C) is 
Hodge generic if s ^ E. In somewhat informal terms we may restate the constancy 
of the Mumford-Tate group over S" \ E by saying that over Hodge generic locus 
the Mumford-Tate group is constant. 

We can also draw conclusions pertaining to the loci in S where we have 
some given collection of Hodge classes. Start with a point sq G ^(C), let xq E S 
be a point above sq, and consider a finite collection of nonzero classes t^^\ for 
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i = 1, . . . , r, in tensor spaces r(m('') = r(S', f (m**))) that are Hodge classes 
for the Hodge structure at the point xq. With notation as above, the locus of 
points in 5^ where all classes t^'^ are again Hodge classes is n • • • n Y{&^). 

The image of this locus in 5* is a countable union of closed irreducible analytic 
subspaces. These components are called the Hodge loci of the given VHS. 

Definition 2.8. Let 7^ be a polarizable Q-VHS over a complex manifold S. A 
closed irreducible analytic subspace Z C S* is called a Hodge locus of H if there 
exist nonzero classes t^^\ . . . , t^''^ in tensor spaces T(m'^*') ~ T(^S, T(m'^*')) such 
that Z is an irreducible component of the image of r(t(i))n---nr(iW) in s. 

Remark 2.9. If we start with a polarizable Z-VHS over a nonsingular complex 
algebraic variety S then by a theorem of Cattani, Deligne and Kaplan, see Corol- 
lary 1.3 in [11], the Hodge loci are algebraic subvarieties of S. See also [81]. 

We note that in the definition of Hodge loci there is no loss of generality to 
consider only a finite collection of Hodge classes. The requirement that a (possibly 
infinite) collection of classes t^*'' are all Hodge classes translates into the condition 
that the homomorphism /i: § — > GL(i/)ffi that defines the Hodge structure factors 
through Mr, where M C GL{H) is the common stabilizer of the given classes. 
Among the classes t*^*^ we can then find a finite subcoUection t^'^^ \ . . . that 
have M as their common stabilizer; so the Hodge locus we are considering is 
defined by these classes. 

3. Special subvarieties of Ag, and the Andre-Oort conjecture 

In this section we discuss the notion of a special subvariety in a given Shimura 
variety. The abstract formalism of Shimura varieties provides a good framework 
for this but it has the disadvantage that it requires a lot of machinery. For this 
reason we shall give several equivalent definitions and we focus on concrete ex- 
amples related to moduli of abelian varieties. In particular the version given in 
Definition 13.101 can be understood without any prior knowledge of Shimura vari- 
eties. 

Further we state the Andrc-Oort conjecture. A proof of this conjecture, 
under the assumption of the Generalized Riemann Hypothesis (GRH), has been 
announced by Klingler, Ullmo and Yafaev. 

3.1. Shimura varieties 

In this paper, by a Shimura datum we mean a pair (G,X) consisting of an 
algebraic group G over Q together with a G(IR.)-conjugacy class X of homomor- 
phisms § — !• Gk, such that the conditions (2.1.1.1-3) of [53] are satisfied. The 
space X is the disjoint union of finitely many connected components. These com- 
ponents have the structure of a hermitian symmetric domain of non-compact type. 
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Associated with such a datum is a subficld E(G,X) C C of finite degree 
over Q, called the reflex field. Given a compact open subgroup K C G(Afin) we 
have a Shimura variety Shi<- (G, X), which is a scheme of finite type over E{G, X), 
and for which we have 

ShK{G,X){C) = G{Q)\{X X G(Afi„)/i^) . 

Fov X G X and 7A' e G(Afin)//^ we denote by [x,-fK] e Shx(G,X)(C) the class 
of (x, jK). 

If K' C A' is another compact open subgroup of G(Afin) we have a natural 
morphism SYik'^k- ShK'iG,X) ^ ShK{G,X). 

Let 7 £ G(Afin). Given compact open subgroups K, K' C G(Afin) with 
K' C 7/C7~i we have a morphism = [-7]: ShK'(G,X) ^ ShK(G,X) that 
is given on C-valued points by T^[x,aK'] = [x^a'^K]. (For 7 = 1 we recover 
the morphism S\yk',k-) This induces a right action of the group G(Afin) on the 
projective limit 

Sh(G,X) := ^ShK(G,X), 

K 

and Shx(G, X) can be recovered from Sh(G, X) as the quotient modulo K. More 
generally, for compact open subgroups Ki, K2 C G(Afin) and 7 6 G(Afin) we have 
a Hecke correspondence from Sh/^ ^ (G, X) to Sh^a (G, X), given by the diagram 

(3.2) Shxi (G, X) ( ^^'"'-"^ shif / (G, X) Sh^, (G, X) , 

where A'' := Ki n 7/^27" ^■ 

3.3. The description of Ag as a Shimura variety 

With notation as in 10.11 let denote the space of homomorphisms /i : § — > 
CSp2g^R that define a Hodge structure of type (—1,0) + (0, —1) on V for which 
±(27ri) ■(()■. V y.V ^ Z(l) is a polarization. The group CSp2g(IR) acts transitively 
on S^g by conjugation, and the pair (CSp2gQ,itig) is an example of a Shimura 
datum. The reflex field of this datum is Q. 

The associated Shimura variety is known as the Siegel modular variety and 
may be identified with the moduli space of principally polarized abelian varieties 
with a level structure. To explain this in detail, we define, for a positive integer m, 
a compact open subgroup Km C G(Afin) by 

AT™ = {7 e CSp(F«)Z,(/)) I 7 = id (modm)}. 

For m > 3 we then have an isomorphism 

P'- -^gJmJ.Q — > Shif^(CSp2g^Q,^g) . 

On C-valued points /3 is given as follows. To begin with, a C-valued point of Ag_[m] 
is a triple (A, A, a) consisting of a complex ppav of dimension g together with a 
level m structure that is symplectic for some choice of a primitive mth root of unity 
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C G C. Write H := Hi{A,'E) for the singular homology group in degree 1 of (the 
complex manifold associated with) A, and let -0 : HxH ^ Z(l) be the polarization 
associated with A; see p.3|) . Choose a symplcctic similitude s: H V. (Even 
though ij} takes values in Z(l) and takes values in Z, the notion of a symplectic 
similitude makes sense.) Via s, the natural Hodge structure on H corresponds 
to an element x G Sjg. Further, we have an identification A[m\ — H/mH, such 
that e^[P,Q) = exp{ip{y,z)/m) if P, Q G A[m](C) correspond to y, z e H/mH, 
respectively. (Note that exp(-0(y, z)/m) is well-defined.) Hence the given level 
structure a can be viewed, via s, as an element jKm € CSp(y (8) (Z/mZ),(/)) = 
CSp(y ® Z, (f>)/Km- The isomorphism /3 then sends (^, A) to the point [x, jKm]- 
For m > 3 the group CSp2g(Q) acts properly discontinuously on S)g x 
CSp2g(Afin)/A'm. The space has two connected components, and if i^^ is 
one of these, the (p{m) components of ^g,[m],c all have the form r\.^+ for an 
arithmetic subgroup F C CSp2g(Q). For m = 1 it is no longer true that CSp2g(Q) 
acts properly discontinuously on S]g x CSp2g(Afin)/iiri, and we should interpret 
the quotient space CSp2g{Q)\{^g x CSp2g(Afin)/ifi) as an orbifold. Alternatively, 
we may take the actual quotient space; this gives us an isomorphism 

CSp2g((Q)\(i0g X CSp2g(Afin)/A-i) ^ Ag(C) 

between Sh^i (CSp2g.Q, iOg) (C) and the set of C-valued points of the coarse moduli 
space. 

Remark 3.4. Let C = C{Vq, (f) denote the symplectic Grassmanian of Lagrangian 
(i.e., maximal isotropic) subspaces of Vq with respect to the form 0. The map 
?)g 'C(C) that sends a Hodge structure y & Sjg to the corresponding Hodge 
filtration Fil° C Vfc is an open immersion, known as the Borel embedding. For an 
arbitrary Shimura datum (G, X) we have, in a similar manner, a Borel embedding 
of X into the C-points of a homogeneous projective variety. 

Because Fil" C Vc is maximal isotropic, the form induces a perfect pairing 
(jj: Fil° X Vc/Fil*^ — !■ C. Via the Borel embedding, the tangent space of S)g at the 
point y maps isomorphically to the tangent space of C at the point Fil°, which 
gives us 

Tyi^^g) ^ Hom^y'"(Fil",l/c/Fil°) 
''^'^^ := {13 : Fif Vc/Fil° | 0(w, I3{v')) = 4>{v' , j3{v)) for aU v' G Fil°} . 

The condition that 0(w, /?(«')) = 4'{v' , f3{v)) means that /3 is its own dual, via the 
isomorphisms Fil° {Vc/Fify and Vc/Fif ^ (Fify induced by 4>; whence 
the notation Hom"^™. 

3.6. Special subvarieties 

There are several possible approaches to the notion of a special subvariety of 
a given Shimura variety. Let us start with the construction that gives the quickest 
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definition. We use the language of Sliimura varieties; tlie example to keep in 
mind is the Sicgel modular variety Sh(CSp2g Q,^g)- After this we shall give some 
alternative definitions that lean less heavily on the abstraet formalism of Shimura 
varieties. The reader who prefers to avoid the language of Shimura varieties is 
encouraged to skip ahead to Definition 13.101 

We fix a Shimura datum (G, X) and a compact open subgroup K C G(Afin). 
If (M, Y) is a second Shimura datum, by a morphism / : (M, Y) — > (G, X) we mean 
a homomorphism of algebraic groups f : M ^ G such that for any y: S ^ Mr 
in Y the composite / o y: § — > Gm is an element of X. The existence of such a 
morphism of Shimura data implies that E{M,Y) contains E{G,X), and / gives 
rise to a morphism of schemes 

Sh(/): Sh(M,y) ^ Sh(G,X)£(M,y) • 

Let 7 S G(Afiii). The image of the composite morphism 

(3.7) Sh(M, Y)c Sh(G, X)c ^ Sh(G, X)c ShuiG, X)c 

is a reduced closed subscheme of ShK{G,X)c- 

Definition 3.8 (Version 1). A closed subvariety Z C Sh/^(G, A')c is called a 
special subvariety if there exists a morphism of Shimura varieties / : (M, Y) — s- 
(G, X) and an element 7 G G(Afin) such that Z is an irreducible component of the 
image of the morphism p.7p . 

For a second approach we fix an algebraic subgroup M C G over Q. Define 
Ym C X a.s the set of all x: § — > Gr in X that factor through Mr C Gr. We give 
Ym the topology induced by the natural topology on X. The group M{R) acts 
on Ym by conjugation. It can be shown (see [33], Section 1.3, or [35], 2.4) that 
Ym is a finite union of orbits under Af (R). We remark that the condition that Ym 
is nonempty imposes strong restrictions on M ; it implies, for instance, that M is 
reductive. 

Definition 3.9 (Version 2). A closed subvariety Z C Sh/^(G, X)c is called a 
special subvariety if there exists an algebraic subgroup M C G over Q, a con- 
nected component Y~^ C Ym, and an element 7 G G(Afi„) such that Z{C) C 
Shj<-(G, X)(C) is the image of Y+ x {jK} C X x G(Afi„)/A' under the natural 
map to ShK{G,X){C) ^ G(Q)\(X x G(Afin)//v). 

For the equivalence of this definition with Version 1 we refer to [44], Propo- 
sition 3.12 or [45], Remark 2.6. 

Our third version of the definition, which in some sense is the most conceptual 
one, describes special subvarieties of Shi<-(G, X) as the Hodge loci of certain natural 
VHS associated with representations of the group G. As we wish to highlight 
the Siegel modular case, we only state this version of the definition for special 
subvarieties of .4g [,„] c for some to > 3. (See, however, Remark l3.1in 
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In order to talk about Hodge loci we need some VHS. The most natural Q- 
VHS on Ag,[m],c to consider is the variation Ti whose fiber at a point (A, A, a) 
is given by Hi{A,Q). (One could also work with the Q-VHS given by the first 
cohomology groups; for what we want to explain it makes no difference.) The 
Hodge loci of this VHS are precisely the special subvarieties. 

Definition 3.10 (Version 3). A closed subvariety Z C ^g,[m],c is called a special 
subvariety if it is a Hodge locus of the Q-VHS H over Ag^[rn].c whose fiber at a 
point (A, A, a) is Hi{A,Q). 

Concretely this means that the special subvarieties are "defined by" the ex- 
istence of certain Hodge classes, i.e., they are the maximal closed irreducible sub- 
varieties of Ag^[m].c oil which certain given classes are Hodge classes. In order to 
make this precise, one has to pass to the universal cover, where the underlying 
local system can be trivialized, as explained in Section [^771 Note that in this case 
the Hodge loci are algebraic subvarieties of ^g.[m], ci this can be shown by proving 
that this notion of a special subvariety agrees with the one given by Version 1 of 
the definition but it also follows from the theorem of Cattani, Deligne and Kaplan 
mentioned in Remark 12.91 

Remark 3.11. Version 3 of the definition of a special subvariety, in terms of 
Hodge loci, can be extended without much difficulty to arbitrary Shimura varieties 
Sh.K{G, X). The variations of Hodge structure one considers are those associated 
with representations of the group G. See for instance |45| . Proposition 2.8, which 
also proves the equivalence with the earlier definitions. We note that, even if one 
is only interested in statements about special subvarieties of Ag, there are good 
reasons to extend this notion to more general Shimura varieties. The formalism 
of Shimura varieties enables one to perform some useful constructions, such as the 
passage to an adjoint Shimura datum, or the reduction of a problem to the case 
of a simple Shimura datum. 

Example 3.12. Let {A, A, a) be a complex ppav of dimension g with a syniplectic 
level m structure, for some m > 3. Let D := End°(A) be its endomorphism 
algebra. We should like to describe the largest closed (irreducible) subvariety 
Z C Ag^[rn\,c that contains the moduli point [A, A, a] G ^g_[m](C), and such that 
all endomorphisms of A extend to endomorphisms of the universal abelian scheme 
over Z. In terms of Hodge classes this means that all elements of D, viewed as 
Hodge classes in EndQ(7Ji(A,Q)), should extend to Hodge classes in the whole 
Q-VHS over Z given by the endomorphisms of the first homology groups. (Note 
that we may pass to Q-coefficients, for if / G End(A) is an endomorphism such 
that some positive multiple nf extends to an endomorphism of the whole family, 
/ itself extends.) 

As before, choose a symplectic similitude s: Hi{A,'Z) V with respect to 
the polarization forms on both sides. This gives Vq the structure of a left module 
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over the algebra D. Let G S)g be the point given by the Hodge structure on 
Hi{A,Z), viewed as a Hodge structure on V via s. Further, let M C CSp2g q ~ 
CSp(Vq, (j)) be the algebraic subgroup given by 

A/ = CSp(yQ,0)nGLD(FQ). 

The homomorphism /iq : § — CSp2g jj factors through A/jj. Conversely, if /i g Sjg 
factors through A'/r then D acts by endomorphisnis on the corresponding abelian 
variety. This means that Z C Sh/f^^ (CSp2g q, iOg)c = ■^g.[m],c is the special 
subvariety that is obtained, with notation as in Definition 13.91 as the image of 
Y^j X {7/im} in Ag^[m],Cj were Y^^ C Ym is the connected component contain- 
ing ho and "fKm G CSp2g(Afiii)/_ftr,„ is the class corresponding (via s) with the 
given level structure a. 

We conclude that the closed subvarieties Z C Ag^[m],c "defined by" the exis- 
tence of endomorphisms, as made precise above, are examples of special subvari- 
eties. 

Remark 3.13. The special subvarieties considered in Example 13.121 are referred 
to as special subvarieties of PEL type; the name comes from the fact that they 
have a modular interpretation in terms of abelian varieties with a polarization, 
given endomorphisms and a level structure. 

In the above example our focus is on the concrete modular interpretation of 
these special subvarieties. For many purposes it is relevant to also have a good 
description of these examples in the language of Shimura varieties; see also pi] . 
Section 4. Here one starts with data {D,*,V,(j)) where D is a finite dimensional 
semisimple Q-algebra, * is a positive involution, y is a faithful (left) Z3-module of 
finite type, and (j>: y x y ^ Q is an alternating form such that 

(3.14) 4>{dv,v') = (l3iv,d*v') for all u, w' e y and d e D. 

Let G = CSp(y, 0) n GLd{V) be the group of D-linear symplectic similitudes 
of V. Next one considers a G°(]R)-conjugacy class X of homomorphisms § Gr 
defining on V a Hodge structure of type (—1, 0) -I- (0, —1) such that ±2ni • (/> is a 
polarization. The pair (G'',X) is then a Shimura datum. 

In Example 13.121 the data {D,V,4>) are given, and we take for * the invo- 
lution on D defined by (|3.14p . For X we take the conjugacy class of the homo- 
morphism ho- By construction, the inclusion G ^ CSp2g defines a morphism of 
Shimura data /: (G'^,X) — >■ (CSp2g,i3g), and the special subvariety Z C ^g,[m],c 
of l3.12l is an irreducible component of a Hecke translate of the image of Sh(G°, X) 

in ShA',„(CSp2g^Q,i3g)c = ^g,[m],C- 

Example 3.15. Let {G,X) be a Shimura datum. If a;: S — > Gr is an element 
of X, we define the Mumford-Tate group of a; to be the smallest algebraic subgroup 
MT:i, C G over Q such that x factors through MT^j^r. 



18 



The Torelli locus and special subvarieties 



Let s = [xj-fK] G ShA'(G, X)(C). Up to conjugation by an element of G(Q), 
the Mumford-Tate group MT^^ C G is independent of the choice of the chosen 
representative (x^jK) for s. Hence we may define the Mumford-Tate group of s 
to be MT^ := MT^. 

A point X £ X is called a special point if MT^; is a torus. A point s G 
Sh/<-(G, X) (C) is a special subvariety of S1ik(G, X)c if and only if MT^ is a torus, 
i.e., if for some (equivalently, any) representative [x, ^K] the point x £ X is special. 

In the Siegel modular variety Ag over C, the special points are precisely the 
CM points, i.e., the points corresponding to ppav {A, A) such that A is an abelian 
variety of CM type; see Mumford [S3], § 2. 

3.16. Basic properties of special subvarieties 

We list a number of elementary properties. 

(a) Hecke images of special subvarieties are again special. More precisely, consider 
a Hecke correspondence as given by (|3.2p . Let Z C ShxiiG, X)c be a special 
subvariety. Write T^{Z) for the image of Sh]^] j(^{Z) under the map [-7]. Then all 
irreducible components of T^{Z) are special subvarieties of Shx2(G,X)c. 

As a particular case, if we have compact open subgroups K' C K C G(Afin) 
and if y C ShK'{G, X)c is a special subvariety, the image of Y in Sh/<-(G, X)c is 
again a special subvariety. Conversely, if Z C Shx(G,X)c is a special subvariety, 
the irreducible components of Sh]^] j^{Z) are special subvarieties of Sh;f/(G,X)c. 
In the study of special subvarieties, these remarks often allow us to choose the 
level subgroup K as small as needed. 

(b) The special points in a special subvariety are dense. If Z C S\yk{G, X)c is a 
special subvariety then the special points in Z are dense for the analytic topology 
on Z{C)] in particular they are Zariski dense. In order to prove this, the essential 
point is to show that Z contains at least one special point. The density of special 
points then follows from the fact that, with notation as in Def. 13.91 the set of special 
points in Ym is stable under the action of and that M(Q) is analytically 
dense in M(M). 

In view of the importance of special points, let us sketch a proof of the 
existence of at least one special point, following [53], § 3. The argument uses the 
fact from the theory of reductive groups that, given a maximal torus T C Gr, the 
G(M)-conjugacy class of T contains a maximal torus that is defined over Q. 

Start with any a;: S Gm in X. Let G C Gm be the centralizer of 
which is a connected reductive subgroup of Gr. (See [77], Lemma 15.3.2.) Choose 
a maximal torus T C G. Because a;(S) is contained in the center of C , we have 
x{§) C T. Hence, if T' C Gr is any torus that contains T then T' centralizes x{S) 
and therefore T' C G. It follows that T is also a maximal torus of Gr. By the 
general fact stated above, there exists an element g e G(]R) and a maximal torus 
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S C G (over Q) such that gTg^^ = 5r. Then gx — Inn(5) o x: S ^ Gr factors 
through Sfi] hence g X is a special point. 

A more refined version of the existence of special points plays a role in the 
theory of canonical models of Shimura varieties. See [H], Theorem 5.1. 

(c) Intersections of special subvarieties are again special. This is easily seen using 
Version 2 of the definition. 

(d) After passage to an appropriate level cover, special subvarieties are locally 
symmetric. Suppose Z C Sh/f(G,X) is a special subvariety. We may then find a 
subgroup K' C K of finite index and an irreducible component Z' C Shx' (G, X) 
of the inverse image of Z such that Z' is a locally symmetric variety. (We shall 
recall the definition of this notion in 14.81 below.) 

Remark 3.17. A situation we often encounter is that we have an abelian scheme 
A ^ T over some complex algebraic variety T (assumed to be irreducible), with 
a principal polarization X: A ^ A*^ and a level m structure a. This gives us a 
morphism t: T ^ Ag^[m].c^ s-^d we should like to know if the scheme-theoretic 
image of r is a special subvariety. (In this situation, the scheme-theoretic image 
is the reduced closed subscheme of ^g.[m] that has as underlying set the Zariski 
closure of the topological image of r.) 

We remark that the answer to this question only depends on A up to isogeny, 
and is independent of the polarization and the level structure. 

Of course, if we change the polarization or the level structure, or if we replace 
A/T by an isogenous abelian scheme, the morphism r is replaced by another 
morphism t' : T ^ but if the image of t is special, so is the image of r'. (For 

simplicity of exposition, we here assume the new polarization is again principal, 
but even this assumption can be dropped.) The reason that this is true is that 
the Mumford-Tate group of an abelian variety A only depends on A up to isogeny, 
and not on any additional structures. 

The conclusion, then, is that one may set up the situation as one finds it 
convenient. We could equip {A, A) with a level structure (possibly after replacing T 
with a cover), allowing us to work with a fine moduli scheme Ag^[m]. Alternatively, 
one could forget about the level structure and consider the morphism T — ;> Ag 
to the moduli stack; in this case we should talk about special substacks of Ag, 
which makes perfectly good sense. Yet another option is to consider the morphism 
T — s> Ag to the coarse moduli scheme. For the question whether the closed image 
of T is special, it does not matter which version we consider. 

Further, since we are only interested in the closure of t{T), we may replace T 
by an open subset, and in fact it suffices to know the generic fiber of A/T. 

Conjecture 3.18 (Andre-Oort, [2], [61], [62]). Let {G,X) be a Shimura datum. 
Let K C G(Afin) be a compact open subgroup and let Z C Shx(G, X)c be an 



20 



The Torelli locus and special subvarieties 



irreducible closed algebraic subvariety such that the special points on Z are dense 
for the Zariski topology. Then Z is a special subvariety. 

An alternative way of stating the conjecture is that, given a set of special 
points & C ShxiG, X)(C) , the irreducible components of the Zariski closure of 6 
are special subvarieties. 

We refer to Noot's Bourbaki lecture for an excellent overview of what 
was known about the conjecture in 2004. 

Klingler and Yafaev [37] , using the work of UUmo and Yafaev [7S| , have an- 
nounced a proof of the Andre-Oort conjecture, assuming the Generalized Riemann 
hypothesis (GRH) for CM fields. 

Theorem 3.19 (Klingler- Yafaev). Assume the GRH holds for all CM fields. Then 
the Andre-Oort conjecture is true. 

There are some special cases of the Andre-Oort conjecture that are known 
to hold without any assumptions on the GRH. See for instance [46], Theorem 5.7, 
later refined by Yafaev in [52], Theorem 1.2 of [55], and Theorem 1.2.1 (with con- 
dition (2)) of [37] ■ In s-ll these cases, however, further assumptions are needed on 
the set of special points. Apart from trivial cases, the only completely uncondi- 
tional case of the Andre-Oort conjecture that we know of, is the main result of 
Andre's paper [3], which proves the conjecture for subvarieties of a product of two 
modular curves, and the more recent extension of this by Pila [53] to arbitrary 
products of modular curves. 

Remark 3.20. As in Remark [3.17[ consider a principally polarized abelian scheme 
{A, A) over some complex algebraic variety T. Again we consider the question 
whether the closed image of the morphism t : T Ag is a special subvariety. 
Suppose that A/T is isogenous to a product Ai x A2 with dim{Ai/T) ~ gi. After 
choosing polarizations (principal, say) on the factors Ai we get morphisms Ti: T ^ 
Ag. . In this situation, if the closure of t(T) in Ag is a special subvariety, the closure 
of Ti{T) in Ag. is special, too. If one believes the Andre-Oort conjecture this is 
clear, and it is in fact not very difficult to show this using our definitions of a 
special subvariety. 

The converse implication does not hold, in general. So, if the closures of 
Ti(T) and T2{T) are special, this does not imply that the closure of t{T) is special. 
Looking at it from the perspective of the Andre-Oort conjecture, suppose that 
for each of the two factors Ai/T separately, we have a Zariski dense collection of 
points in T(C) at which the fiber of Ai/T is of CM type. Then it is not true, in 
general, that there is a dense set of points at which the fibers are simultaneously 
of CM type. (For a concrete example, take T to be an open part of \ {0, 1}, 
let Ai /T be a family of elliptic curves that is not isotrivial, and take for A2 the 
same family with a suitable shift in the parameter, i.e., such that A2^t = ^i.t+e 
for some fixed e G C.) 
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4. Special subvarieties in the Torelli locus 

Throughout this section we work over C. 

Conjecture 4.1 (Coleman, |16| . Conjecture 6). Given g > A there are only finitely 
many non-singular projective curves C over C, up to isomorphism, of genus g and 
the Jacobian Jc is a CM abelian variety. 

As we shaU discuss below, the conjecture is known to be false for g < 7, so a 
corrected version of the conjecture should have as an assumption that 5 > 8. 

Both Coleman's conjecture 14. II and the Andre-Oort conjecture 13. 181 were in- 
spired by the analogy with the Manin-Mumford conjecture, now a theorem of 
Raynaud. For proofs of the Manin-Mumford conjecture, see [TT], [72]; for an easy 
proof see [7D]. The analogy between this conjecture and the Andre-Oort conjec- 
ture is discussed for instance in [17], Section 6 and in (SS], Section 3. As we 
shall discuss next, the Andre-Oort conjecture also has important implications for 
Coleman's conjecture. 

Expectation 4.2 (Oort, [52], § 5). For large g (in any case g >8), there does not 
exist a special subvariety Z (Z f\g with dim(Z) > 1 such that Z C Tg and Z r\T° 
is nonempty. 

Note that the assumption that Z nT° is nonempty implies that this inter- 
section is open and dense in Z . 

Remark 4.3. The condition that Z meets T° is important. Indeed, it is easy to 
sec that for any g > 2 there exist special subvarieties Z C Ag of positive dimension 
with Z C Tg. For <? < 3 this is clear, as Tg = Ag is special. Assume g > 3. Choose 
a base variety T and a stable curve C7 — s- T of genus 3, such that the closure of 
the image of T in A3 is a special subvariety of positive dimension d. Let J ^ T 
be the Jacobian, A its canonical principal polarization. Next take an elliptic curve 
E with complex multiplication, and let ^ be its principal polarization. Then for 
every t e T(C) the moduli point G Ag(C) of the ppav (Jt, A) x {E,y)3-^ Hcs 
in the closed Torelli locus Tg, as it is the Jacobian of the curve that is obtained 
from Ct by attaching to it a tail of g — 3 copies of E. Moreover, it is not hard 
to see that the closure of the set of points is a c?-dimensional special subvariety 
of Ag. In this way we can produce many positive dimensional special subvarieties 
in Tg for any g > 2. 

Remark 4.4. Assume we have an integer g for which Expectation 14.21 holds. 
Assume furthermore that the Andre-Oort Conjecturc l3.18l is true. Then Coleman's 
Conjecture 14.11 is true in genus g. In fact, let CM(Tg) C T(C) be the set of all 
CM Jacobians of dimension g. If this set is infinite, its Zariski closure in Ag has at 
least one irreducible component Z of positive dimension, which by 13.181 is special, 
contradicting 14.21 Hence and [XT51 together implv HTI 
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Remark 4.5. For g > 3 wc know that Tg itself is not a special subvaricty, and in 
fact there is no special subvariety S C Ag that contains Tg. (Note that Tg Ag 
for g > 3.) The simplest argument we know for this is to use information about 
the geometric monodromy. For convenience, let us pass to moduli spaces with a 
level m structure, for some m > 3. Write Tg.[m] C ^g.[m] for the Torelli locus 
in Ag_[m]. Over ^g,[m] we have the natural Q-VHS H considered before (see just 
before Definition I3.10|) . and the assertion that Tg is not contained in any special 
subvariety S' C Ag is implied by the fact that the generic Mumford-Tate group of 
the restriction of H to 7^,[m] is the full group CSp2g,Q- 

Write H' for the restriction of H to 7^,[m]- Choose a Hodge generic base 
point b € Tg^im], let H be the fiber of H' at b, and let ip be the polarization form 
on H. Further, let M C CSp2g_Q = CSp{H,-ip) be the Mumford-Tate group at b, 
which is the generic Mumford-Tate group of H'. Then M contains Gm • 1. On the 
other hand we have a monodromy representation 

p:^i(rg,[„,],6)^GL(if), 

and since we have passed to a level cover, p factors through Sp(iJ, ip). By a result of 
Deligne, see [52], Proposition 7.5, the image of p has a subgroup of finite index that 
is contained in Af (C). So we are done if we can show that the image of p is Zariski 
dense in Sp(i?, "0). This can be seen, for instance, using transcendental methods. 
In fact, the homomorphism p: T^i{Tg,[m] , b) — > Sp(-ff, ip) may be identified with the 
natural homomorphism Fg — >■ Sp(iJ, ip) from the mapping class group in genus g 
to the symplcctic group, and it is a classical result that this homomorphism is 
surjectivc. (See for instance [5], Chap. 15, especially § 3.) 

The argument given here is entirely based on information about the geometric 
monodromy, and there are other loci in Tg to which the same reasoning applies. 
As an example, if Hg C Tg is the hyperelliptic locus, the image of the geometric 
monodromy representation on Hg is again dense in the symplcctic group Sp(iJ, ip); 
see [T], Theorem 1. By the above argument, it follows that there is no special 
subvariety 5 C Ag that contains Hg. 

4.6. A modified version of Coleman's Conjecture 

In [13] we find a modified version of 14. II which docs hold, at least conditionally, 
for every 5 > 3. A 5-dimensional abelian variety A, say over C, is said to be a 
Weyl CM abelian variety if L := End°(A) is a field of degree 2g over Q whose 
Galois closure has degree 2^ ■ gl over Q. It can be shown that, in a suitable sense, 
most CM abelian varieties are of this type. 

Tlieorem 4.7 (Chai and Oort). Assume the Andre-Oort Conjecture 13.181 to be 
true. Then for every 5 > 3 the number of Weyl CM points in T° is finite. 
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See [13, 3.7. We remark that for g > 4 wc do not know any example 
of a Jacobian of Weyl CM type. In connection with this, note that for a non- 
hypereUiptic curve C of genus g{C) > 1 with Aut(C) 7^ {id} the Jacobian Jc does 
not give a Weyl CM point. 

4.8. Results of Hain 

In his paper [33] , Hain proved some results inspired by 14.21 Though the 
results are conditional, they point in an interesting direction, as they suggest 
that ball quotients should play a special role. (The open unit ball in C" is the 
symmetric space SU(n, 1)/U(n).) It should be pointed out that all non-trivial 
examples known to us, in genus at least 4, are indeed ball quotients; see the next 
sections. 

In order to state Hain's results, let us first recall that a complex algebraic 
variety 5" is called a locally symmetric variety if there is a semisimple algebraic 
group G over Q, a maximal compact subgroup K C and an arithmetic 

subgroup r C G{Q), such that the symmetric space G{M.)/K is hermitian and 
S = T\G{M.)/K. As we have seen in 13.161 special subvarieties are (at least after 
passage to a level cover) locally symmetric. 

If ^1 = ri\Gi(M)/Xi and S2 = T2\G2iR)/K2 are locally symmetric vari- 
eties, a morphism / : 6*1 — >■ S'2 is called a map of locally symmetric varieties if it 
is induced by a homomorphism of algebraic groups Gi — >■ G2 over Q. 

Hain calls a locally symmetric variety S good if it has no locally symmetric 
divisors. Further, he mostly restricts his attention to locally symmetric varieties S 
for which the corresponding Q-group G is almost simple (i.e., G'^'^ is simple); for 
the problems that interest us this is no loss of generality. In case G is almost 
simple, it can be shown that S is not good only if the Q-rank of G is < 2 and 
if the non-compact factors of Gr are all of the form SO(n, 2) or SU(ri,, 1), up to 
isogeny. 

Define a locally symmetric family of abelian varieties to be a principally 
polarized abelian scheme X S such that 5 is a locally symmetric variety and 
the corresponding morphism S* — >■ is a map of locally symmetric varieties. By a 
locally symmetric family of curves we mean a curve G — ?► 5 of compact type such 
that the corresponding relative Jacobian J ^ S' is a locally symmetric family of 
abelian varieties. 

Theorem 4.9 (Hain). Let tt: G —?' S be a locally symmetric family of curves 
that is not isotrivial. Assume the Q-group that gives S is almost simple. Assume 
further that either tt is smooth, or S is good and the generic fiber of tt is smooth. 
Then S is a quotient of a complex n-ball. 

See [33], Theorem 1. 

It should be realized that for applications to Coleman's conjecture, the as- 
sumptions of the theorem are too strong. The main problem is that a special 



24 



The Torelli locus and special subvarieties 



subvaricty Z as in 14.21 only gives us (possibly after passing to a level cover) a lo- 
cally symmetric family of abelian varieties J ^ Z such that the geometric generic 
fiber is a Jacobian. We may find a dominant morphism Z' ^ Z such that the 
puUback J' Z' is the relative Jacobian of a smooth curve C ^ Z' but in general 
it is not possible to do this with Z' an open part of a locally symmetric variety. 
See [33] . in particular Proposition 8.3. 

Under weaker assumptions, Hain proves a second result. 

Theorem 4.10 (Hain). Let A ^ S be a locally symmetric family of abelian va- 
rieties such that the morphism p: S ^ Ag is not constant and factors through the 
Torelli locus Tg. Write 

gdcc j^g £ S{C) I As is the Jacobian of a singular curve,} 
S* := 5(C) \ 5'^'='= , 

5''° {.s G S{C) I As is the Jacobian of a hyperelliptic curve.} 

We assume the Q-group that gives S is almost simple, S is good, and S* =^ 9. Then 
either S is a ball quotient, or else g > 3, each component of 5"^°^^ has complex 
codimension > 2 in S , the family does not lift to a locally symmetric family of 
curves, and S* D S^^'^ is a non-empty divisor in S* , which moreover for g > 3 is 
nonsingular. 

See [33], Theorem 2. 

The proofs of Hain's results rely on a rigidity property of mapping class 
groups, which is a special case of the theorem of Farb and Masur in |31j . 

De Jong and Zhang [T^ have pushed Hain's results further, based on the 
observation that the hyperelliptic locus in kg \ k'^'^^ is affine. 

Theorem 4.11 (de Jong and Zhang). With assumptions as in Theorem \4-10[ 
either S is a ball quotient, or S'^'^'^ has codimension < 2 in S , or the Baily-Borel 
compactification of S has a boundary of codimension < 2. 

Corollary 4.12. Let S G Ag, with g > ^, be a Hecke translate of a Hilbert modular 
subvaricty of Ag, i.e., S is a special subvaricty of PEL type obtained from a totally 
real field F of degree g. Then S is not contained in Tg. 

The Corollary was proved in [19], except when g = 4 and F contains a 
quadratic subfield. That case was settled by Bainbridge and MoUer in [5]. 

In addition to the results discussed here, there are several other results in- 
spired by 14.21 Among these are papers of Moller, Viehweg and Zuo; see [H], {13] 
and [80], and the recent paper [4] by F. Andreatta. These results support the 
Expectation 14.21 
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4.13. Results of Ciliberto, van der Geer and Teixidor i Bigas 

In [15] and [M], Ciliberto, van der Geer and Teixidor i Bigas have obtained 
some interesting results about the number of moduli of curves whose Jacobians 
have nontrivial endomorphisms. As we shall discuss below, when combined with 
the results of Hain and de Jong-Zhang, these results can be used to obtain some 
restrictions on the special subvarieties of PEL type that are contained in the Torelli 
locus. 

As always in this section we work over C. 

Theorem 4.14 (Ciliberto, van der Geer and Teixidor i Bigas, [15]). Let Z C Mg, 

for g > 2, be an irreducible closed subvariety. Let be an algebraic closure of the 
function field C(Z), let C/Q be the curve corresponding to the geometric generic 
point of Z , and assume that the Jacobian Jc/^ has the property that Z C End( Jp). 
Then dim(Z) < 2g — 2, and the intersection of Z with the hyperelliptic locus in Mg 
has dimension at most g. 

Note that in this theorem it is not assumed that the image of Z in Ag is a 
special subvariety. 

In addition to the result as quoted here, the authors have some finer results 
about the case when dim(Z) > 2g — 3. For instance, they show that if dim(Z) = 
2g — 2 then either C is a cover of a non-constant elliptic curve E over 17, or g = 2 
and End° (Jc) is a real quadratic field. (The case dim(Z) = 2g — 3 is analyzed by 
Ciliberto and van der Geer in |14].) 

4.15. Excluding certain special subvarieties of PEL type 

One may use the results we have discussed to obtain restrictions on the special 
subvarieties Z C Ag of PEL type that can be contained in Tg with Z n T° ^ 0. 
(We do not, however, see a way to apply such arguments to arbitrary special 
subvarieties.) Our result is as follows. 

Theorem 4.16. Consider a a special subvariety S C Ag of PEL type, arising from 
PEL data {D, *, V, <f) as in Remark l3.13l with AYm^iV) = 2g. 

(i) Suppose D = F is a totally real field. (Albert Type L) If g > A then S is 
not contained in Tg . 

(ii) Suppose D is a quaternion algebra over a totally real field F that splits at 
all infinite places of F. (Albert Type IL) If g > 8 then S is not contained in Tg. 

For Albert's classification of the possible endomorphism algebras of abelian 
varieties we refer to [53], Section 21. 

Proof. In both cases it follows from the results of [7S] that D equals the endomor- 
phism algebra of the abelian variety corresponding to the geometric generic point 
of S. Hence the generic abelian variety in our family is geometrically simple; in 
particular, if 5 C Tg then S" n T° 7^ 0. 
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First suppose D = F is a totally real field. Let e — [F : Q], which is an 
integer dividing g. Then S arises as a quotient of a product of e copies of the 
Siegel space Sjh with h ^ g/e. This gives 

h{h + l) gjg + e) 
Theorem 14. 141 gives the inequality g{g + e)/2e < 2g — 2, so we find 

q Q 
e > — > - . 

- 3g - 4 3 

As e divides g, either g is even and e — g/2 or e ^ g. However, for 5 > 4 the 
case e = g is excluded by Corollarv l4.12l Assume then g > 4 is even and e = g/2; 
in this case dim(S') = 3g/2. The assumptions in Hain's theorem 14. 101 are satisfied 
and S is not a ball quotient. Hence we obtain that the intersection with the 
hyperelliptic locus is a nonempty divisor and therefore has dimension (3g/2) — 1. 
This contradicts Theorem 14. 141 

Next we consider the case where D is a quaternion algebra over a totally real 
field F such that Z^OqR is isomorphic to a product of e [i^ : Q] copies of A/2(M). 
In this case 2e divides g and 5 is a quotient of a product of e copies of with 
h = g/2e; so dim(5') = e • h{h + l)/2 and S is not a ball quotient. The boundary 
in the Baily-Borel compactification has codimension g/2 > 2. By Theorem 14. Ill it 
therefore suffices to show that 8'^°^ cannot have codimension < 2. 

There are two possibilities for the geometric generic point of a component 
of S*^""^. Either the corresponding abelian variety is isogenous to a product Xi x 
X2, where Xi and X2 both have an action by an order in D. Straightforward 
calculation gives that in this case the codimension is at least e ■ {h ~ 1). Using 
the relation g — 2eh we find that for g > 8 the codimension is > 2. The other 
possibility is that there is a subfield F' C F and a quaternion algebra D' with 
center F' such that D = F ®f' D' . With e' = [F' : Q] the codimension in this 
case equals (e — e') • h{h + l)/2. Straightforward checking of the possibilities shows 
that for g > 8 this is greater than 2. □ 

It seems that similar arguments will also work for the Albert Type HI, and 
even for Type IV we expect that we can obtain some non-trivial conclusions. We 
have not yet pursued this. 

4.17. The Schottky problem 



Expectation l4.2l is of course intimately related to the Schottky problem, which 
is the problem of characterizing which ppav (A, A) are Jacobians of curves. There 
are several solutions or conjectural solutions of this problem. We refer to the 
overview papers [7], |17| . |79| for an introduction to this beautiful topic. It seems 
that none of the (conjectural) solutions discussed in these papers can be directly 
applied to problems such as Coni ecture 14 . 1 1 or Expectation 14.21 Though for both 
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sides — algebraic curves and their moduli on the one hand, abelian varieties and 
special subvarieties in the moduli space on the other hand — we have many tech- 
niques and results at our disposal, it is difficult to find a language, or a set of 
techniques, using which both sides can be described simultaneously. This is a 
difficulty that we believe lies at the heart of the matter. 

5. Examples of special subvarieties in the Torelli locus 

In this section we discuss examples of special subvarieties S* C Tg of positive 
dimension, with g > 4 and S nT° ^ 0. The examples we consider arise from 
families of cyclic covers of P^. Throughout this section we work over C. 

We first look at a concrete example, following [TB]. (The example was already 
given in [75].) 

Example 5.1. Consider the family of curves given by 

=x{x-\){x-t), 

where t G C \ {0, 1} is a parameter. The complete and regular model Ct of this 
curve is a cyclic cover of with group /is, with total ramification above 0, 1, i 
and oo. The Hurwitz formula gives x = 5 • — 2 + 4 • (5 — 1) = 6 so g[Ct) = 4. In 
the moduli space A4 the corresponding family of Jacobians gives a 1-dimensional 
subvariety, whose closure we call Z C A4 . Clearly these Jacobians admit an action 
by ^[Cs], the ring of integers of the cyclotomic field F := Q[C5]. 

Consider the special subvariety S" C A4 containing Z that is defined by this 
action of '^[(5]- More formally, fix a base point b G C \ {0, 1} and choose a 
symplectic similitude s: i?i(C6,Z) ^ V as in 13.31 Via s, the Hodge structure 
on Hi{Cb,Q) corresponds to a point y & i)4 and we obtain the structure of an 
i^- vector space on Vq. Let M C Gq = CSp(F, 0)q be the algebraic subgroup 
obtained as the intersection of G with GLp{Vq). With notation as in Def. 13. 9[ 
y lies in Ym- If we choose the base point b such that the Jacobian Ji, is not of 
CM type (which is certainly possible) then there is a unique connected component 
Y'^ C Ym containing y, and 5 C A4 is the special subvariety obtained as the image 
of this under the quotient map S)^ CSp(y, (j))\S)4 ^ A4(C). 

As we shall show, dim(S') = 1. Assuming we know this, we conclude that Z, 
which is also 1-dimensional and is contained in S, contains an open dense subset 
of S, in which case it follows from property (b) in 13.161 that there are infinitely 
many values of t such that the Jacobian of Gt is of CM type. 

In order to calculate the dimension of S, we need to know how F acts on 
the tangent space of the Jacobian Jt at the origin. More precisely, To{Jt) has the 
structure of a module over the ring F^qC = JJ^. C. Hence we obtain a direct 
sum decomposition T(){Jt) ~ (BaT{<7). Let Ua- denote the C-dimension of T{a). 
These multiplicities do not depend on t. By using the polarization, one can 
show that na- + n^ = 2g/ip(5) = 2 for all ct, where ct denotes the complex conjugate 
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of a. With this notation wc have H^'' = 811(71^, jt-ct) and dim(S') = ^n^rig, 
where the sum is taken over a set of representatives of the complex embeddings 
of F modulo complex conjugation. See [75j . Theorem 5, and see below for further 
details. 

As To(Jt) is canonically dual to i/"(Ct,w), we can calculate the multiplici- 
ties n^r by writing down a basis of regular differentials on Ct- (In fact, as we shall 
explain below the Chevalley-Weil formula gives a much quicker method.) In the 
example at hand, if Pi, for i G {0, 1, t, oo}, is the unique point above i, we have 

div(x) - 5Po - 5Poo , div(y) = + A + - 3Poo , 

and 

div(da;) = 4Po + 4Pi + 4Pt - 6Poo • 
As a basis of regular differentials we find 

dx dx dx xdx 

yZ yi yi yi 

The weights of the first two are dual, whereas the last two forms have the same 
weight. The conclusion, then, is that there is one pair (cr, tr) with multiplicities 
(2, 0), and one pair where the multiplicities are (1, 1). Hence dim (5) = l-l + 2-0 = 
1, and we conclude that S* = Z is a special subvariety contained in T4 with 
S* n T4 ^ 0. In particular this proves that Coleman's conjecture 14. II does not hold 
for g = 4. 

Remark 5.2. In the above example, we do not know a method to decide for which 
values of the parameter t the Jacobian of Ct is a CM abelian variety. The same 
remark applies to the examples we shall discuss next. 

5.3. Families of cyclic covers 

To obtain further such examples, the idea is to fix an integer m > 2, an integer 
A'^ > 4, and monodromy elements ai, . . . ,ajv in Z/rnZ; then we consider cyclic 
covers of with group /^m, branch points ti, . . . ,tf^ in and local monodromy 
exp(27riaj/m) S about tj. If the branch points arc all in A^. this cover is given 
by the affine equation 

(5.4) = (x - ^i)''^ [x - hT' ■■■{x- t^r" , 

with ( G acting by (.t, y) H> {xX ■ v)- Varying the branch points ti gives us a 
family of curves, and it turns out that for certain choices of the data involved, the 
corresponding family of Jacobians traces out a special subvariety in /Kg. 

Write a = (ai, . . . , oat). The triple {m,N,a) that serves as input for our 
construction has to satisfy some conditions. As already indicated, we want m > 2, 
as taking m = 1 is clearly of no interest. Next, we require that A > 4. The 
reason is that from our family we obtain an (A^ — 3)-dimcnsional subvariety in the 
moduli space (see below), and we are interested in special subvarieties of positive 
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dimension. Wc assume that ^ (mod m) for all i, and that ai + ■ ■ ■ + = 
(mod to). This means that the ti are branch points and that there are no further 
branch points. Further we need to assume that the elements ai generate the group 
Z/toZ for otherwise the (smooth projective) curves given by (|5.4p are reducible. 

We can find an open subscheme T C {F^)^ , disjoint from the big diagonals, 
and a smooth proper curve f : C ^ T such that the fiber of / at a point t ~ 
{ti, . . . ,tN) in r(C) with all ti in (C) is the complete regular curve given by (|5.4p . 
The genus of these curves is given by 

(5 5) g = l+ (^~^)"^~E»^igcd(«""^) 

The relative Jacobian J ^ T then defines a morphism t : T ^ Ag, and we define 
Z{'m,N,a) C Ag as the closure of the image of r. Note that since we are only 
interested in the closure of t(T) there is no need to specify exactly which open 
subscheme T C (A^)^ we choose; see Remark [3. 171 

We call two triples (m, A^, a) and {m' , N' ,a') equivalent if to = to' and 
N ~ N' and if the classes of a and a' in (Z/toZ)^ are in the same orbit under 
{JjjmX)* X &N- Here (Z/toZ)* acts diagonally on (Z/mZ)^ by multiplication, 
and the symmetric group &n acts by permutation of the indices. The closed 
subvariety Z{m,N,a) C Ag only depends on the equivalence class of the triple 
(to, N, a). 

The morphism r: T -> Ag factors through the quotient of T modulo the 
action of the group PGL2(C) x ©at, with PGL2(C) = Aut(P^) acting diagonally on 
(P^)^ and 6 jy acting by permutation of the diagonals. (Without loss of generality 
we may assume T C (Pi)^ is stable under PGL2(C) x ©at.) The subvariety 
Z(rn,N,a) C Ag has dimension N — 3. Note that we may also fix three of the 
branch points to lie at 0, 1 and oo, as we did in Example l5.ll this has the effect of 
replacing T with a closed subvariety of dimension — 3 on which the morphism (f> 
is generically finite. For instance, the example considered in 15.11 corresponds to 
the triple {m,N,a) = (5,4,(1,1,1,2)). 

The Jacobians Jt in our family come equipped with an action of the group 
ring Z[/im]. Let S{firn) C Ag be the special subvariety containing Z(rn, N,a) that 
is defined by this action. (In order to make this precise we follow the recipe given 
in Example 15. 11 ) In all cases we have the inequality 

(5.6) A^-3 < dimS'(Ai,„), 

and if we have N — 3 ^ dim S{firn) then the conclusion is that Z(m, N, a) ~ S{firn) 
is a special subvariety in the Torelli locus that meets the open Torelli locus. (If 
A'^ — 3 < dim5(/i„i) then a priori we know nothing; see l5.16l below.) The question 
is therefore how to calculate the dimension of S{^m)- Before we discuss this in 
general, let us look at another example. 
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Example 5.7. Consider the family of curves given by 

if = x{x - l){x - t) . 

With notation as above we have m = 9 and = 4, with local monodromy about 
the branch points given by a = (1, 1, 1, 6). 

The complete regular model Ct is a cyclic cover of with group /ig. The 
points 0, 1 and t are totally ramified, so we have unique points Pq, -Pi, and Pt above 
them. There are three points Pio^ and Pio'' above oo, each with ramification 
index 3. The Hurwitz formula gives x = 9 " ~2 + 3- 8 + 3- 2=12sog( = 7, which 
agrees with (|5.5p . 

The Jacobian Jt contains as an isogeny factor the Jacobian of the curve C[ 
given by = x{x — l)(x — t). By a similar calculation, C[ has genus 1. Its 
Jacobian J[ is an elliptic curve with complex multiplication by Zf^a], and the 
family of Jacobians J[ is isotrivial over \ {0, 1}. (Note that in the given equation 
for C[ there is no ramification over oo, so C[ is geometrically isomorphic to the 
curve given by = v{v — 1). More explicitly, let T — ;> P^ \ {0, 1} be the cyclic cover 
of degree 3 given by the equation = t{t — 1). After base change to T the family 
of curves becomes isomorphic to the constant curve defined by s'^ = u(w — 1) 
via the isomorphism given by s = fj,u/ {x — t) and w = (1 — t)x/ (x — t). ) 

Let Jf™ (the "new part") be the quotient of Jt modulo J[; this is an abelian 
variety of dimension 6 on which we have an action by Z[^g]. In order to determine 
the dimension of the special subvariety in Ag given by this action, we again calcu- 
late the multiplicities of the action on the tangent space. In this case, we already 
know in advance that there will be one non-primitive character of /xg occurring in 
H^{Ct,Lo). We have 

div(a;) = 9Po - W^^^ - 3P^^ - SP^^) , div(y) = Pq + Pi + P* - P^^) - P^^ - P^^ , 
and 

div(da;) = 8Po + 8Pi + 8Pt - AP^^ - 4P^^ - AP^^ . 
As a basis of regular differentials we find 

dx dx dx dx dx xdx xdx 

4" ' ^ ' ' 7" ' s" ' 7~ ' s~ ■ 

y y y y y y y 

As predicted, this gives one non-primitive character of /ig (corresponding to dx / y^); 
for the rest we find one complex conjugate pair with multiplicities (1, 1), and two 
complex conjugate pairs for which the multiplicities are (2,0). The special sub- 
variety in Ag given by the action of Q[C9] with this collection of multiplicities has 
dimension l-l + 2- + 2- = l. It follows that the Jf™ trace out a dense open 
subset of this special subvariety. As the original Jacobians Jt arc isogcnous to 
J'^ X Jf^^ with J( an isotrivial family with fibers of CM type, we conclude that the 
family of Jt traces out in A7 a (dense subset of a) special subvariety. As before. 
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this implies that there are infinitely many values of t for which Jt is of CM type. 
(The t for which this happens are even analytically dense in P^(C).) 

Remark 5.8. Let (M, Y) be a Shimura datum. The dimension of the associated 
Shimura varieties SIik {M,Y) only depends on the structure of the real adjoint 
group Mg*^. Indeed, the dimension of the Shimura variety equals the complex 
dimension of the space Y, and the components of Y can be described as hermitian 
symmetric domains associated with the connected Lie group AI'^'^{M.)~^ , where the 
superscript "+" denotes the identity component for the analytic topology. In 
particular, if M^'^ = Qi x ■■■ x Qi is the decomposition of M^'^ as a product of 
K-simple groups, the dimension of ShxiM, Y) can be calculated as a sum S{Qi) + 

h(5(Qi), where the contribution S{Qi) of the factor Qi can be looked up in 

Table V. (Caution: the dimensions given there are the real dimensions.) The cases 
most relevant for our discussion are the following. 

(1) 5{Qi) = if Qi is anisotropic, i.e., if Qi{M.) is compact; 

(2) 5{Q,) = h{h + l)/2 if g, ^ PSp2„,h; 

(3) <5(g,) =pqif g, ^PSU(p,q). 

5.9. Calculating the dimension of 5(/i„i) 

We return to the general situation of a family of curves / : C — ?■ T associated 
with the data (m, TV, a). Our next goal is to calculate the dimension of the special 
subvariety 5(/i„i) C Ag that contains Z = Z{m, N, a). Choose a Hodge- generic 
base point b G r(C), choose a symplectic similitude s: ifi(Cf,,Z) ^ as in 13.31 
and let y & S)g he the point corresponding to the Hodge structure on i7i(Cf,,Z), 
viewed as a Hodge structure on V via s. 

The Jacobian J ^ T comes equipped with an action of the group ring 
Z[/Xm]. The Hodge classes we want to have over S{fj,m) are these endomor- 
phisms, which means we are in the situation of Example 13.121 The algebraic 
group M C CSp(T42, </>) we need to consider is given by 

M := CSp(Fq,0) n GLq[^„.](Fq) . 

We can calculate the dimension of S{fj,m) via a deformation argument. In 
other words, if Ym C is as defined just before Definition 13.91 we calculate the 
dimension of the tangent space of Ym at te point y. 

The Hodge structure on Vq is of type (—1, 0) + (0, —1), and the polarization 
gives us a symplectic form ip: Vq x Vq ~> Q(l)- As discussed in Remark 13.41 
the Hodge filtration Fil° — V^'~^ C Vc is a Lagrangian subspace, so we have an 
induced isomorphism (f): Fil° ^ Vc/Fil°. The tangent space of S)g at the point y 
is given by p.Sp . 

The extra structure we now have is an action of the algebra D := Q[yLtm] 
on Vq. Let d ^ d denote the involution of D induced by the inversion in the 
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group firn- The form 4> has the property that (f)(dv,v') = (j){v,dv') for all d G I? 
and V, v' € Vq. 

The D-action on Vq induces on V^^^'^ and V^'~^ the structure of a module 
over the ring 

I? ®Q C = Yl C. 

Correspondingly, we have decompositions 

Fc/Fil" = ^e"^'" = e ^c:(„^ and Fif = V^^~'= 

The involution of Dq obtained by linear extension of the involution d ^ d ex- 
changes the factors C indexed by the classes n and —n. It follows that the perfect 
pairing (j>: Fil° x {Vc/Y\^) C restricts to perfect pairings 

-^-^H x^c:(-„)^C. 

For n e Z/mZ, define 

dn := dime ^cXn) ■ 

Note that d(omod7n) = Oj i-e-, Vc,(omodm) = (0), as the y^tm-invariant subspace 
in V is the first homology of the base curve P-'^, which is zero. We shall see in 
Proposition lS. 1 ll below how to calculate the c?„ in terms of the given data (to, N, a). 

The tangent space TyiYM) C Ty{S)g) is the C-subspace of elements f3 e 
Hom"y'"(Fil°,Fc/Fil°) that are Z^c-hnear. Any such /3 can be written as f3 = 
^ /3„ , where the /?„ : V^, — >■ V^^^^^ are C- linear maps that satisfy 

(5.10) ^n{v,P-niv')) = 4>^n{v',Mv)) for all v G V^'-^] and v' G V°--^\y 

If n ^ —n (mod to), this last condition gives a duality between /3_„ and /3„; this 
means the linear map /3„ can be chosen arbitrarily and /3_„ is determined by /?„. 
The situation is different if n = —n (mod to). Of course, this only occurs (with 
n ^ 0) if TO = 2fc is even. In this case we have a perfect pairing 0^ : V^'^^^ x ^£(^1^) ~^ 
C, and gives 

/3.GHom^--(y,°;-;-Vt:?.?) 

- <W ^ ^cTw I M^,Pk{v')) = Mv',Mv)) for all v, v' G ^^"'(I.)} • 
(Cf. (|3.5p .) We find that the dimension of Ty{YM) equals 

dk-idk + l] 



d-ndn + 



if TO = 2fc is even; 



z)/{±i} |0 if TO is odd. 

2n^0 (mod m) 

As the final step in the calculation we need to calculate the dimensions dn in 
terms of the given triple {m,N,a). The result is as follows. 
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Proposition 5.11 (Hurwitz, Chcvallcy-Wcil). The dimensions dn ■= dime 
are given by dn = if n = (mod m) and 

(5.12) ^" = -l + H\~^) (modm), 

i=i \ / 

where {x) ~ x — \x\ denotes the fractional part of a number x. 

We note that V^'~^ is naturally isomorphic to the space H'^{Ci,, Q,^) ol global 
differentials on the curve. The given formula is then a special case of a classical 
result by Chevalley and Weil about the structure of H°{Cb, fl^) as a representation 
of the Galois group ^m, which in the cyclic case is already due to Hurwitz. See 
|52| Section 3 for a modern proof. Another proof, using the holomorphic Lefschetz 
formula, can be found in [74) . Lemma 1.6b. 

Putting everything together, the dimension of S{iirn) is given by the following 
result. 

Proposition 5.13. Consider a triple {m,N,a) as in Section \ 5.3[ Then the di- 
mension of the special subvariety 5(/i„i) that contains Z{m, N, a) is given by 



dim S{Hjn) = ^ d^ridn + 



if m ^ 2k is even; 



if m is odd, 

where the sum runs over the pairs ±n in Z/mZ with 2n ^ (mod m), and where 
the dn are given by (|5.12p . 

Remark 5.14. Instead of using a tangent space computation, we may calculate 
the dimension of S{/j,m) by analyzing the real adjoint group Mk; see Remark 15.81 
and IIHI . It can be shown that 



M^''^l[PSV{dn,d^n) 



f ^P2(ifc R if = 2fc is even; 
{1} if m is odd, 

where the first product runs over the pairs ±n in Z/mZ with 2n ^ (mod m). 
5.15. An inventory of known special subvarieties in the Torelli locus 

Now that we have an explicit formula for the dimension of S{firn) in terms of 
the given data, it is easy to check, in each given example, if in the inequality (|5.6p 
we have an equality. Recall that if N—3 = dim S{prn), we conclude that Z{m, N, a) 
is dense in the special subvariety S{firn) C Ag, in which case it follows that the 
family of Jacobians J ^ T (as in section 15. 3|) contains infinitely many different 
Jacobians of CM type. 

In order to illustrate how well this works, let us redo Example 15.11 We have 
(m, a) = (5, 4, (1, 1, 1, 2)). The Hurwitz-Chevalley-Wcil formula gives 

d, = -l + 3.(^\ + (^\=2. 
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and in a similar way we find 

d2 



1 , 



1 . 



0, 



which agrees with the basis of regular differentials we have found. Proposition l5.13l 
gives dimS'(^m) = 1 = — 3, and as before we conclude that Z{m,N,a) is a 
special subvariety. 

Using a computer it is not hard to do a systematic search for triples (m, N, a) 
with — 3 = dim5(/i„i)- Up to equivalence (as defined in Section [5.3p . one 
finds twenty such triples. They are listed in Table [TJ Moreover, it was proven 
by Rohde in [73j that these are the only triples, up to equivalence, for which 
— 3 = dim S{^j,m)- 

Table 1 . Examples of special subvarieties in the Torelli locus 





genus 


m 


TV 


a 


(1) 


1 


2 


4 


(1,1,1,1) 


(2) 


2 


2 


6 


(1,1,1,1,1,1) 


(3) 


2 


3 


4 


(1,1,2,2) 


(4) 


2 


4 


4 


(1,2,2,3) 


(5) 


2 


6 


4 


(2,3,3,4) 


(6) 


3 


3 


5 


(1,1,1,1,2) 


(7) 


3 


4 


4 


(1,1,1,1) 


(8) 


3 


4 


5 


(1,1,2,2,2) 


(9) 


3 


6 


4 


(1,3,4,4) 


(10) 


4 


3 


6 


(1,1,1,1,1,1) 





genus 


m 


TV 


a 


(11) 


4 


5 


4 


(1,3,3,3) 


(12) 


4 


6 


4 


(1,1,1,3) 


(13) 


4 


6 


4 


(1,1,2,2) 


(14) 


4 


6 


5 


(2,2,2,3,3) 


(15) 


5 


8 


4 


(2,4,5,5) 


(16) 


6 


5 


5 


(2,2,2,2,2) 


(17) 


6 


7 


4 


(2,4,4,4) 


(18) 


6 


10 


4 


(3,5,6,6) 


(19) 


7 


9 


4 


(3,5,5,5) 


(20) 


7 


12 


4 


(4,6,7,7) 



5.16. Excluding further examples 

In the discussion so far, the argument is based on the fact that Z{m, N, a) C 
Ag is visibly contained in the special subvariety S'(/im), and we look for exam- 
ples where the two have the same dimension. If in some given case we find that 
dim Zlm, N, a) ~ N—3 < dim S{firn), this docs not, a priori, imply that Z{m, N, a) 
is not special. Put differently, in addition to the endomorphisms in Z[/^,„], there 
might be Hodge classes in our family of Jacobians J — T that we just happen not 
to see. Even in individual examples, it is usually not so easy to exclude this. 

One method to do this was given by de Jong and Noot in [T5], Section 5; 
they prove there that for m > 7 not divisible by 3 the family of curves given by 
= x{x — l)(a; — t), which in our language corresponds to the triple {m, N, a) = 
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(m, 4, (1, 1, 1, m — 3)) , docs not give a special subvariety. The method is based on 
results of Dwork and Ogus in [27] . 

Extending this to arbitrary families, it was proven in [48] that the twenty 
examples we have found are the only ones such that the image Z{m,N,a) C Ag is 
a special subvariety. 

Theorem 5.17. Consider data (rn,N,a) as in 15.31 Then Z{m, N,a) C Ag is a 
special subvariety if and only if {m, N, a) is equivalent to one of the twenty triples 
listed in Tabic [TJ 

Remark 5.18. Examples (6), (8), (10), (11), (16) and (17) in Tablcfflwcrc given 
by Shimura in [76]. Examples (10), (11) and (17) were given, in a language that 
is closer to the present paper, by de Jong and Noot in [TH]; they also explained 
the relevance of such examples for Coleman's conjecture. (The fact that these 
examples already occurred in [76] was recognized only later. In connection with 
this, note that [73] appeared more than twenty years before Coleman stated his 
conjecture.) 

In retrospect, it is surprising that the complete list of examples was obtained 
only recently. Example (19) was found by one of us in 2003; see [S^- AH twenty 
examples were found by Rohde in [TJ (Example (2) is somewhat hidden there, 
but see op. cit. Corollary 5.5.2) and independently by one of us, with the help of 
a small computer program. See 16.81 below for some further examples, coming from 
families of covers with a non-cyclic Galois group. 

It should be mentioned that there is some connection between these examples 
and the theory of Deligne and Mostow in [5B] , [JT] ; see also Looijenga's overview 
paper [39]. The relation is that, in the setting of l5.3i the family of Jacobians J ^ T 
has a "new part" (cf. Example 15. 7p . and that the monodromy of the corresponding 
VHS can be described as the monodromy on a space of hypergeometric functions. 
In this context there is an easy criterion for the arithmcticity of the monodromy 
group; see [55], Proposition 12.7 or [33], Theorem 4.3. As far as we know there is, 
however, no easy way to obtain from this, and the resulting tables in [26| and |51| . 
a classification result such as in Theorem 15.171 Apart from the fact that the 
Deligne-Mostow arithmcticity criterion only applies under some condition on the 
local monodromy elements (condition (INT) of [23] i P- 25) that in our situation is 
not always satisfied, it only gives us information about certain direct summands 
of the VHS that we want to study, and as already explained in Remark 13.201 we in 
general need more. 

6. Some questions 

In this section, with the exception of 16. 91 and 16.121 the base field is C. Most 
questions below could also be formulated over an algebraic closure of Q. 
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Question 6.1. How do we construct curves for which the Jacobian is a CM abelian 
variety? If we have a special subvariety in Tg that intersects T°, the existence 
of CM Jacobians follows from property (b) in 13.161 Note that in such cases we 
typically have no control over which fibers in our family are the CM fibers; cf. 
Remark O 

On the other hand, we could look for curves that have many automorphisms. 
For instance, the Fermat curves X" + y" + Z" = in have Jacobians of CM 
type. Similarly, cyclic covers of with 3 branch points have CM Jacobians. For a 
given g > 2 there are only finitely many such covers of genus however. Further 
such examples are given in [64| . 5.15. 

In [64], a curve C is called a curve with many automorphisms if the defor- 
mation functor of the pair (C, Aut(C)) is a 0-dimensional scheme. In many cases 
such a curve has a CM Jacobian. 

These are the only methods known to us to construct, or to prove the exis- 
tence of, CM Jacobians. 

Question 6.2. Do we know the existence of, or can we construct, a curve C of 
genus at least 4 such that Aut(C) ~ {id} and such that the Jacobian of C is an 
abelian variety of CM type? 

Question 6.3. Does there exist g > i and a special subvariety Z C Ag contained 
in the Torelli locus Tg such that the geometric generic fiber over T gives an abelian 
variety with endomorphism ring equal toTL ? We do not know a single example. 

Remark 6.4. In 14.21 we have seen an expectation about the (non-)existence of 
special subvarieties in the Torelli locus. In order to make the question more precise, 
consider, for g g Z>o the set 

ST{g) {special subvarieties Z C Tg with dim(Z) > and 2" n T° 7^ 0} 

of special subvarieties of positive dimension, contained in the Torelli locus, and 
not fully contained in the boundary of Tg. The expectation is that for g ^ Q we 
have5r(g) = 0; seelH 

We would like to classify all pairs (g, Z) with Z G ST{g). For g = 2 and g = 3 
we have Tg = Ag and in this case every special subvariety of Ag is of PEL type; 
see [SD]. Hence in this case we can classify all pairs (g, Z), up to Hecke translation, 
by listing all possible endomorphism algebras. We know that ST{g) ^ for all 
g < 8. However, already for g = 4 we do not have a good description of iST(4). It 
seems very difficult to describe ST{g) for arbitrary g. 

6.5. Non-PEL Shimura curves for .g = 4 

In [53] , § 4, Mumford shows there exist 1-dimensional special subvarieties 
Z C A4 that are not of PEL type. The abelian variety corresponding to the 
geometric generic fiber of Z has endomorphism algebra Z. The curves Z are 
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complete. Note that T4 C A4 is a closed subvaricty of codimension one, which by 
a result of Igusa |35] is ample as a divisor. Hence we see that Z n T4 7^ 0. 

Question 6.6. Is there a "Mumford curve" Z C A4 that is contained in T4 ? 

If Z C A4 is a 1-dimensional special subvariety of the type constructed by 
Mumford, the abelian variety corresponding to the geometric generic point of Z 
has endomorphism ring Z; hence if Z C T4 then Z meets T|. 

The examples constructed by Mumford, and generalizations thereof, have 
been studied in detail by Noot in [57] and [58]. In particular, [58], Section 3 
contains a detailed analysis of the possible CM points on the special curves Z C A4 
constructed by Mumford. In particular, it is shown there that the (geometric) CM 
fibers are either absolutely simple or are isogenous to a product E x Y with E 
an elliptic curve, Y an abelian threefold, and End°(i?) isomorphic to a subfield 
of End*'(y). It might be possible to use this to get some non-trivial information 
related to the above question. In connection with what was discussed in 14.61 let 
us note that there are no Weyl type CM fibers in these families. 

Question 6.7. For which g > 2 does there exist a positive dimensional subvariety 
Z C Tg with Z n T° 7^ 0, such that the abelian variety corresponding with the 
geometric generic point of Z is isogenous to a product of elliptic curves ? 

This question was stimulated by results in |38| . which say that under more 
restrictive conditions such a family does not exist for large g. 

Question 6.8. In Section[5]we have seen examples of special subvarieties Z cTg 
with ZC\T° 7^ arising from families of cyclic covers of . Can one obtain further 
such examples by taking non-cyclic covers, or from a family of covers of another 
base curve? 

To make this more precise, consider a (complete, nonsingular) curve B over C 
of genus /i, and let Q he a, finite group. Define a group 11 = n(ft,, N) by 

n := (ai, . . . ,a/i,^i, . . . ,/3/i,7i, . . . ,7Ar | • • • [ah.Ph] • 7i ' • ' 7JV = l) ■ 

Given ti, . . . , tjv m -B, fix a presentation 

7ri(s\{ii,...,tjv}) -^n, 

and fix a surjective homomorphism -0: 11 -» f/ such that ^'{"/i) 7^ 1 for all i ~ 
1, . . . ,N. Correspondingly, we have a Galois cover Ct —?' B with group Q, branch 
points ti,...,tpf in B, and with local monodromy about ti given by the ele- 
ment ipili)- Varying the branch points, we get a family of curves C ^ T, for 
some open T C B^ . The corresponding family of Jacobians gives a moduli map 
T — !> Ag; denote the image by Z° , with Zariski closure Z C Tg C Ag. Having 
set the scene in this way we can ask for which choices of the data involved Z is a 
special subvariety of positive dimension. 
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(a) Are there examples with non-cyclic Galois group such that Z <Z Ag is a 
special subvariety of positive dimension? 

(b) Are there examples where B is not a rational curve, and such that Z G Ag 
is a special subvariety of positive dimension? Note that if Z is special, the 
Jacobian of B is a CM abelian variety. 

As for question (a), we do have some examples that are obtained from families 
of covers of with a non-cyclic abelian Galois groups. The examples presently 
known to us are listed in Table [D In these examples we consider families of cov- 
ers of with Galois group of the form A = (Z/miZ) x (Z/m2Z) with mi\m2, 
with iV > 4 branch points, and with local monodromy about the branch points ti 
given by an iV-tuple a = (ai,...,a7v) in A^ . This gives rise to an (N — 3)- 
dimensional closed irreducible subvariety Z[m,N,a), where now m = (mi,m2). 
For the Jacobians Jt in our family we have a (generally non-injective) homomor- 
phism Q[A] — > End'^( Jt); this defines a special subvariety S{m) C Ag of PEL type 
with Z{m, N, a) C S{m). We calculate dim(S'(m)), and if this dimension equals 
iV — 3 then wc conclude that Z{m, N, a) is special. 

Table 2. Examples of special subvarieties in the Torelli locus (continued) 





genus 


group 


N 


a 


(21) 


1 


(Z/2Z) 


X (Z/2Z) 


4 


((1,0), (1,0), (0,1), (0,1)) 


(22) 


3 


(Z/2Z) 


X (Z/4Z) 


4 


((1,0),(1,1),(0,1),(0,2)) 


(23) 


3 


(Z/2Z) 


X (Z/4Z) 


4 


((1,0),(1,2),(0,1),(0,1)) 


(24) 


4 


(Z/2Z) 


X (Z/6Z) 


4 


((1,0),(1,1),(0,2),(0,3)) 


(25) 


4 


(Z/3Z) 


X (Z/3Z) 


4 


((1,0),(1,0),(1,2),(0,1)) 


(26) 


2 


(Z/2Z) 


X (Z/2Z) 


5 


((1,0), (1,0), (1,0), (1,1), (0,1)) 


(27) 


3 


(Z/2Z) 


X (Z/2Z) 


6 


((1,0),(1,0),(1,1),(1,1),(0,1),(0,1)) 



Let us give some further details concerning Example (24). In this case the 
covers Ct — >■ P^ are branched over 4 points ti, . . . , ^4. There are 6 points above ti, 
each with ramification index e ~ 2. Likewise, there are 2 points above t2, both 
with e = 6, there are 4 points above ts, each with e = 3, and finally there are 
6 points above ti, each with e = 2. The Hurwitz formula gives x = ^24 + 6-1 + 
2-5-h4-2-h6-l = 6, so5 = 4. The group ring Q[A], with A = (Z/2Z) x (Z/6Z), 
is isomorphic to x ^(Cs)^. Accordingly, the Jacobians Jt split, up to isogeny, 
as a product of eight factors. The simple factors of correspond with the 

Gal(Q/Q)-orbits in Hom(yl, Q*), which in this example may be identified with the 
set of complex characters p £ IIom(A, C*) taken modulo complex conjugation. 
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Given a complex character p: A — > C*, the corresponding factor Jp t of Jt can 
be described as follows. We consider the cover Hp^t ■ Cp^t P^ with group p{A), 
branched only above the points ti, with local monodromy p{ai) about t;. Note 
that p{ai) may be the identity element of p{A), in which case ti is not a branch 
point of TTp.t. Also note that TTp^t is a cyclic cover, which brings us back to the 
situation considered in Sectional Then Jp^t is the new part of the Jacobian of Cp^t- 
With this description, it is easy to verify that: 

• there are five pairs (p, p) for which Jp t = 0; this includes the four real 
characters, for which p = p, 

• there are two pairs for which Jp^t is an elliptic curve with CM by an 
order in Q(C3), 

• there is one pair for which Jp t is 2-dimensional, carrying an action 
by an order in Q(C3); varying t these give a family that is isogenous to the 
family of Jacobians of Example (5) in Table [1] 

It follows from this description that the special subvariety S{m) that contains 
Z(rn, N,a) is 1-dimensional, and therefore Z{m, a) = S{m) is special. 

It should be noted that Example (24) is a sub-family of the family given in 
Example (14). To see this, let Dt be the quotient of Ct modulo the action of 
{1} X (Z/6Z), and factor nt: Ct^ as 

We have Dt = P^. The cover qt has group (Z/6Z) and is branched above five 
points, namely the unique point <2 of Dt above t2, the two points ^3^1 and ^3^2 
above ^3, and the two points ^4,1 and ^4,2 above <4. The local monodromy about 
these points is given by the 5-tuple (2, 2, 2, 3, 3) in (Z/6Z)^, which gives exactly the 
data of Example (14). The sub-family considered here is given by the constraint 
that the five branch points on Dt = P^ do not move freely but form three orbits 
under the action of Z/2Z on Dt. 

6.9. The Serre-Tate formal group structure and linearity properties 

Let m > 3. On [„j](C) we have a metric, obtained from the uniformization 
by the Siegel space f)^; see 13.31 In [JS] it is proven that an irreducible algebraic 
subvariety Z C .^g,[m] is a special subvariety if and only if Z is totally geodesic 
and contains at least one special point. This may be viewed as a characterization 
of special subvarieties in terms of linearity properties. 

This characterization has a nice arithmetic analogue, by which it was in fact 
inspired. Recall that if fc is a perfect field of characteristic p > and x S Ag{k) 
corresponds to a ppav {A, A) over k such that A is ordinary^ the formal completion 
of -4g,[jn] a-t the point x has a canonical structure of a formal torus over the 
ring of Witt vectors W{k); see [36] or [40], Chap. 5. Using this we can again give 
meaning to the notion of "linearity". Let us elucidate this. Consider a closed 
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irreducible algebraic subvariety Z C Ag\^rn\.Fi where _F is a number field. Let Z 
denote the Zariski closure of Z inside .^g.[m] over Op. If some ordinary point x 
as above lies in Z{k)^ the formal completion of -E at a; gives a formal subscheme 
3x C 21a;. It was shown by Noot in [SB] that if Z is a special subvariety, the 
components of the formal subscheme 5x C 'Qlx over W{k) are translates of formal 
subtori of 21^; over torsion points. At the cost of excluding finitely many primes 
of Op this may be sharpened to the conclusion that the components of the formal 
subschemes S^; are formal subtori; see also [iB], Theorem 4.2(ii). 

The converse of Noot's result was proven in [46]; the result is that if Z C 
-^g,[m],F is a closed irreducible algebraic subvariety such that for some ordinary 
point X € Zik) some component of "^x C %x is a translate of a formal subtorus 
of 21a: over a torsion point, Z is a special subvariety. These results again give 
a characterization of special subvarieties in terms of linearity properties. It was 
shown in |45j that the result over C may reformulated in terms of formal group 
structures, in a way that makes the analogy between the two situations even 
clearer. 

These results lead us to investigate the structure of the Torelli locus T g C 
hg (or its analogue with a level structure) locally near an ordinary point x in 
characteristic p. The identity section of the formal torus 21^: gives a lifting of the 
ppav {A, A) over fc to a polarized abelian scheme (^'^™, A*^™) over the ring of Witt 
vectors W{k). This lifting is called the Serre-Tate canonical lifting of {A, A). Note, 
however, that if {A, A) is the Jacobian of a curve, the canonical lifting {A'^^''^, A'^™) 
over W{k) need not be a Jacobian; see [57], [BB]. This leads to another view 
on Expectation 14.21 Indeed, suppose we have a special subvariety Z C Tg with 
Z n T° 7^ 0. Then Z is defined over some number field F, and, with notation 
as above, we can find ordinary points x G Z{k), for finite fields fc, such that 
the corresponding ppav (^, A) is the Jacobian of a curve C/fc, and such that 
the formal completion ^x C 21^ is a union of formal subtori (or even just a formal 
subtorus). In this situation, the canonical lifting (A'^^''^, A''™) is again the Jacobian 
of a curve, which is a highly nontrivial fact. Indeed, the main results of Dwork and 
Ogus in j27j are based on the observation that in general, already the first-order 
canonical lifting, over the ring W2{k) of Witt vectors of length 2, is no longer a 
Jacobian. 

For the next two questions, let m > 3 be an integer, and fix a genus g that 
is large enough, at least g > 7. 

Question 6.10. Let Tg,[m],z, C Ag\m\,i denote the scheme-theoretic image of 
the Torelli morphism A^gjm] ~^ ^g.[m] o'^*^'" Spec(Z). Let fc be a perfect field of 
characteristic p > 0, and suppose x £ 7^,[m].z(fc) is an ordinary point, i.e., the 
corresponding abelian variety is ordinary. Is it true that the formal completion 
'Zx C 2tx of Tg^[m],'z o,t the point x does not contain a formal subscheme 3, flat 
overW{k), of positive dimension, such that is a formal subtorus of%x? 
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A positive answer to this question would confirm Expectation 14.21 It should 
be noted, however, that what we ask here is stronger than what we need for 14.21 
The difference is that in 16.101 we do not require the formal subscheme 3 to be 
algebraic, i.e., to be the formal completion of an algebraic subvariety of Ag_[m] 
passing through the point x. The interesting point, however, is that Question l6.10l 
only depends on the formal completion of the Torelli locus at a single ordinary 
point. One might expect that, in terms of the "linear structure" provided by the 
Serre-Tate structure of a formal torus on Sl^; , the Torelli locus 1^ C 21a; should be 
highly non-linear. 

Over C we have a question that is similar in spirit. 

Question 6.11. Does the Torelli locus Tg^[m],c -4g,[m],c contain any totally 
geodesic subvarieties of positive dimension? 

Again this question is stronger than what is needed for Expectation 14. 2[ as 
we do not require the totally geodesic subvariety to be algebraic. (In addition we 
should ask that the subvariety contains at least one CM point.) 

Let us mention the paper [41) . in which Moller investigates algebraic curves 
in A4g over C that are totally geodesic with respect to the Teichmiiller metric 
(these are called Teichmiiller curves), such that the image of this curve in Ag is 
a special subvariety. It is shown in j41j that for g = 2 and <? > 6 there are no 
such curves. For g ~ 3 and .g = 4 there is precisely one example, corresponding to 
Examples (7) and (12) in Table [ij 

6.12. An analogy 

We can view the boundary of Ag in a compactification as the locus of degen- 
erating abelian varieties, but one can also view, working over Zp, say, the space 
Ag ® Fp as a boundary of Ag <^Zp. In this last case the abelian variety does not 
degenerate, but the p-structure does change. These two points of view have strik- 
ing similarities. Often geometric questions are settled by studying properties at 
the boundary, and then to lift back to the interior of the moduli space considered. 

Here we want to draw attention to the analogy between the results in [57] 
and the results of [32] and |3]. In the first case Dwork and Ogus study ordinary 
Jacobians in positive characteristic and their Serre-Tate canonical lifts to charac- 
teristic zero. In analogy with this, Fresnel and van der Put [32] and Andreatta [4] 
study liftings of Jacobians of degenerate curves. In both cases the authors show 
that, for g > 3; these liftings in general do not lie in the Torelli locus. In particular, 
Andreatta explains in [l] that special subvarieties in Ag have a linear structure at 
the boundary in a toroidal conipactification of Ag , and he shows that the closure 
of the Torelli locus is not linear at the boundary. 
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